Biology in Bloom: Implementing Bloom's Taxonomy to Enhance Student Learning in Biology
Abstract
We developed the Blooming Biology Tool (BBT), an assessment tool based on Bloom's Taxonomy, to assist science faculty in better aligning their assessments with their teaching activities and to help students enhance their study skills and metacognition. The work presented here shows how assessment tools, such as the BBT, can be used to guide and enhance teaching and student learning in a discipline-specific manner in postsecondary education. The BBT was first designed and extensively tested for a study in which we ranked almost 600 science questions from college life science exams and standardized tests. The BBT was then implemented in three different collegiate settings. Implementation of the BBT helped us to adjust our teaching to better enhance our students' current mastery of the material, design questions at higher cognitive skills levels, and assist students in studying for college-level exams and in writing study questions at higher levels of Bloom's Taxonomy. From this work we also created a suite of complementary tools that can assist biology faculty in creating classroom materials and exams at the appropriate level of Bloom's Taxonomy and students to successfully develop and answer questions that require higher-order cognitive skills.
INTRODUCTION
Most faculty would agree that academic success should be measured not just in terms of what students can remember, but what students are able to do with their knowledge. It is commonly accepted that memorization and recall are lower-order cognitive skills (LOCS) that require only a minimum level of understanding, whereas the application of knowledge and critical thinking are higher-order cognitive skills (HOCS) that require deep conceptual understanding (Zoller, 1993). Students often have difficulty performing at these higher levels (Zoller, 1993; Bransford et al., 2000; Bailin, 2002). In the past decade, considerable effort has been directed toward developing students' critical-thinking skills by increasing student engagement in the learning process (Handelsman et al., 2004). An essential component of this reform is the development of reliable tools that reinforce and assess these new teaching strategies.
Alignment of course activities and testing strategies with learning outcomes is critical to effective course design (Wiggins and McTighe, 1998; Sundberg, 2002; Ebert-May et al., 2003; Fink, 2003; Tanner and Allen, 2004; Bissell and Lemons, 2006). Students are motivated to perform well on examinations; therefore, the cognitive challenge of exam questions can strongly influence students' study strategies (Gardiner, 1994; Scouller, 1998). If classroom activities focus on concepts requiring HOCS but faculty test only on factual recall, students quickly learn that they do not need to put forth the effort to learn the material at a high level. Similarly, if faculty primarily discuss facts and details in class but test at a higher cognitive level, students often perform poorly on examinations because they have not been given enough practice developing a deep conceptual understanding of the material. Either case of misalignment of teaching and testing leads to considerable frustration on the part of both instructor and student. Though considerable attention has been given to changing our classrooms to incorporate more active-learning strategies, not enough attention has been placed on how to better align assessment methods with learning goals. Indeed, one of the most significant ways to impact the quality of student learning is through the improvement of our assessments (Entwistle and Entwistle, 1992).
How can we better assess our assessment methods? One approach is to use Bloom's Taxonomy of cognitive domains (Bloom et al., 1956), hereafter referred to as “Bloom's.” Bloom's is a well-defined and broadly accepted tool for categorizing types of thinking into six different levels: knowledge, comprehension, application, analysis, synthesis, and evaluation. A revised version of Bloom's (Anderson et al., 2001) further subcategorizes the original taxonomy and converts the different category titles to their active verb counterparts: remember, understand, apply, analyze, create, and evaluate. Bloom's has been used widely since the 1960s in K-12 education (Kunen et al., 1981; Imrie, 1995) but has seen only limited application in selected disciplines in higher education (Demetrulias and McCubbin, 1982; Ball and Washburn, 2001; Taylor et al., 2002; Athanassiou et al., 2003).
Although Bloom's lends itself to wide application, each discipline must define the original classifications within the context of their field. In biology, Bloom's has been used to design rubrics for evaluating student performance on introductory biology exams (Bissell and Lemons, 2006), develop formative assessment questions at the appropriate cognitive level (Allen and Tanner, 2002), and inform course design (Allen and Tanner, 2007). Nonetheless, there is significant need for more comprehensive assessment tools that undergraduate biology instructors can easily use to assess student learning, guide development of teaching strategies, and promote student metacognition in the biological sciences.
We have developed the Blooming Biology Tool (BBT; Table 1), which can be used to assess the Bloom's Taxonomy level of questions on biology-related topics. The BBT evolved out of a study we were asked to participate in that required us to rank more than 600 biology exam questions from a wide variety of sources including MCAT, GRE, and AP biology exams, as well as introductory biology and first-year medical school courses (Zheng et al., 2008). Here we present a detailed description of the BBT and complementary materials for use by college and university faculty and students. We also highlight how we implemented the BBT and associated learning activities in a variety of educational settings. We found the BBT a useful guide for faculty in diagnosing students' aptitudes and creating new assignments to help students develop critical-thinking skills. Our students used the BBT to create more challenging study questions and self-identify the skill levels that they find the most demanding.
Knowledge1 | Comprehension1 | Application1 | Analysis | Synthesis | Evaluation | |
---|---|---|---|---|---|---|
LOCS2 | LOCS2 | LOCS2 HOCS3 | HOCS3 | HOCS3 | HOCS3 | |
Key skills assessed | IDENTIFY, RECALL,list, recognize, or label | DESCRIBE or explain in your own words, re-tell, or summarize | PREDICT an outcome using several pieces of information or concepts; use information in a new context | INFER; understand how components relate to each other and to the process as a whole | CREATEsomething new using/combining disparate sources of information | DETERMINE/CRITIQUE relative value; determine merit |
General examples of biology exam questions | Identify the parts of a eukaryotic cell; identify the correct definition of osmosis | Describe nuclear transport to a lay person; provide an example of a cell signaling pathway | Predict what happens to X if Y increases | Interpret data, graphs, or figures; make a diagnosis or analyze a case study; compare/contrast information | Develop a hypothesis, design an experiment, create a model | Critique an experimental design or a research proposal; appraise data in support of a hypothesis |
Type of question | ||||||
Labeling | X | X | X | |||
Fill-in-the-blank | X | X | X | X | ||
True-false | X | X | X | X | ||
Multiple-choice | X | X | X | X | X | |
Short answer | X | X | X | X | X | X |
Essay | X | X | X | X | X | X |
Characteristics of multiple-choice questions | Question only requires information recall. Possible answers do not include significant distracters4 | Question requires understanding of concept or terms. Possible answers include significant distracters4 | Question requires prediction of the most likely outcome given a new situation or perturbation to the system | Question requires interpretation of data and selection of best conclusion | N/A: If provided with choices, students only differentiate between possible answers rather than synthesize a novel response | Question requires assessment of information relative to its support of an argument |
DEVELOPMENT OF THE BLOOMING BIOLOGY TOOL
In developing the BBT, we first established a basic rubric that drew extensively on previous interpretations of Bloom's as it relates to biology (Allen and Tanner, 2002; Ebert-May et al., 2003; Yuretich, 2003; Bissell and Lemons, 2006). Through research and discussion, we agreed that the first two levels of Bloom's (knowledge and comprehension) represent lower orders of cognitive skills (Zoller, 1993). We considered the third level of Bloom's, application, to be a transition between LOCS and HOCS. The three remaining categories (analysis, synthesis, and evaluation) are true HOCS but are not necessarily hierarchical, meaning that a question categorized as evaluation does not always require analytical and synthesis abilities, but may require mastery of the lower three levels (knowledge, comprehension, and application). While ranking questions, we found it helpful to “check-off” each level of Bloom's required to successfully answer the question. For example, a question rated at the analysis level would require knowledge (facts), comprehension (understanding of facts), application (predicting outcomes), and analysis (inference). Each question was ranked at the highest level of Blooms' taxonomy required for its solution.
The level of Bloom's that is assessed by a given type of exam question depends highly on what information is provided to the student and which inferences or connections the student must make on his or her own. It is equally important to consider the level of information previously provided through classroom instruction i.e., if students are explicitly given an answer to an analysis question in class and then given that same question on an exam, then that question only requires recall (Allen and Tanner, 2002). We would argue that labeling of diagrams, figures, etc., cannot assess higher than application-level thinking as this question-type, at most, requires students to apply their knowledge to a new situation. However, fill-in the blank, true-false, and multiple-choice questions can be designed to test analysis-level skills. It is nevertheless challenging to develop fill-in-the-blank questions that require higher than application-level thinking, but we have provided one such example (Supplemental Material A; Virology). Further, whereas multiple-choice questions can be designed to assess evaluation skills if they require students to determine relative value or merit (e.g., which data best support the following hypothesis), multiple-choice questions cannot assess synthesis-level thinking as all the answers are provided, eliminating the need for students to create new models, hypotheses, or experiments on their own. Many resources exist to assist faculty in designing high-quality, multiple-choice questions (Demetrulias et al., 1982; Udovic, 1996; Brady, 2005), and we have provided a list of some of these resources (Supplemental Material B).
To differentiate between Bloom's levels, we found it useful to take one particular topic (e.g., cell biology) and develop a series of increasingly challenging exam questions representing the various levels of Bloom's. In developing these multi-level questions, we considered what a student must know or be able to do in order to answer the question. For example, if the student needed to recall factual information and then be able to describe a process in his/her own words, we considered that question to test comprehension. We have provided examples for three different subdisciplines of biology: cell biology, physiology, and virology, (Supplemental Material A). A similar approach was taken by Nehm et al. for the subdisciplines of ecology and evolution (Nehm and Reilly, 2007).
We also found that science questions posed unique challenges to our rubric as they dealt with science-specific skills (e.g., graphing, reading phylogenetic trees, evaluating Punnett squares and pedigrees, and analyzing molecular biology data). To address this, we selected several of these science-specific skills and created examples or descriptions of question-types that would assess mastery at each level (Table 2. Through this process and extensive discussion of our work, we were able to better define and categorize the different types of questions that are typically found on biology exams. To assist us in developing the rubric, we each independently ranked approximately 100 life science exam questions and then extensively discussed our analyses to reach consensus. The BBT reflects the progression of our insights into how to adapt a general assessment method to the discipline-specific skills inherent to biology. We subsequently independently analyzed another 500 questions; statistical analysis of our rankings based on the BBT revealed high interrater reliability (agreement of at least two of the three raters over 91% of the time; [Zheng et al., 2008]).
Knowledge1 | Comprehension1 | Application1 | Analysis | Synthesis | Evaluation | |
---|---|---|---|---|---|---|
LOCS2 | LOCS2 | LOCS2 HOCS3 | HOCS3 | HOCS3 | HOCS3 | |
Calculations | Equation provided and variables identified “plug and chug” | Understand/define components and variables of a given equation | Solve word problems by selecting correct formula and identifying appropriate variables | Solve word problem and infer biological significance or implication | Create an equation that describes the relationship between variables | Evaluate a computational solution to a problem or assess the relative merit(s) of using a specific mathematical tool to solve a particular problem |
Concept maps | Structure provided, student fills in the missing linking phrases or concepts that are provided | Structure provided with concepts filled in, student generates linking phrases to describe relationships | Student creates the structure, concepts and linking phrases provided | Student creates structure, concepts are provided, student generates linking phrases to describe relationships and must link two different domains or maps together | Student creates structure and generates concepts and linking terms, map must be sufficiently complex | Student evaluates existing concept maps based on established criteria/rubric |
Diagnoses | Identify or list variables found in patient history, vital signs, and/or clinical test results. Know which physiological problem each named disease represents (e.g., Graves' disease, hyperthyroidism) | Define each variable. Define the presenting signs and symptoms of each disease | Given a set of clinical variables, identify the relevant variables and make a diagnosis | Given a set of clinical variables and a diagnosis, determine which other possible diseases (differential diagnoses) need to be ruled out | Given a set of clinical variables and a diagnosis, determine the next clinical test that needs to be performed to confirm the diagnosis | Given a set of clinical variables and a diagnosis, evaluate the evidence supporting the diagnosis and provide the patient with a second opinion |
Graphing | Identify the parts of graphs and recognize different types of graphs (e.g., identify the X axis, identify a histogram) | Describe the data represented in a simple graph | Draw a graph based on a given set of data; predict outcomes based on data presented in graph | Read and interpret a complex graph having multiple variables or treatments and explain biological implications of data | Create a graphical representation of a given biological process or concept | Assess the relative effectiveness of different graphical representations of the same data or biological concept |
Hardy- Weinberg analyses | Given the Hardy-Weinberg (HW) equation define terms: p2, q2, 2pq; If given p+q = 1 and p = 0.7, calculate q | Describe the assumptions of the Hardy-Weinberg equation and its use as a null hypothesis What does 2pq represent in the Hardy-Weinberg equation? (HW equation not given) | Determine the expected number of homo-zygous recessive individuals in a population if the recessive allele is represented in 30% of that population (HW equation not given) | Determine if the following population is in HW equilibrium: 100 individuals of which 37 are SS, 8 are ss, and 55 are Ss. Defend your answer | Create a new version of the Hardy-Weinberg equation that incorporates 3 alleles | Analyze Chi-square resuts to weigh predicted evolutionary flux |
Molecular techniques | Identify what is being measured by a molecular technique (e.g., Northern analysis measures relative RNA levels in a given cell or tissue) | Understand what the results of a molecular technique indicate (e.g. the intensity of a band on a Northern blot indicates relative expression of a specific mRNA in the cell type or tissue from which the RNA was obtained) | Draw the expected results you would obtain from a given molecular technique or state which technique could be used to solve a novel problem (e.g., draw the banding pattern you would expect if you analyzed a protein complex containing a 55 kDa protein and a 35 kDa protein by SDS-PAGE) | Interpret the raw data obtained from a molecular technique, including the interpretation of controls and how to normalize data (e.g. interpret the results of a RT-PCR gel analysis by comparing relative expression of experimental genes to a standardized control gene) | Design an experiment using a given molecular technique to test a hypothesis (e.g., design an experiment using Northern analysis to test the hypothesis that transcription factor A regulates expression of gene B) | Assess relative merit of using two different molecular approaches to address a particular hypothesis (e.g., discuss the relative merits of using chromatin immunoprecipitation vs. electrophoretic mobility shift assay to test the hypothesis that a protein binds directly to the promoter of a particular gene |
Phylogenetic tree/cladogram | Given a cladogram, circle the root, nodes, or monophyletic groups | Describe the relationship of sister taxa in a cladogram | Given four cladograms, identify which one is different and describe the evolutionary relationships that make it different | Given a set of taxa for which all but one in a pictured tree exhibit a synapomorphy, infer the evolutionary history of one of the taxa. With respect to that same synapomorphy, discuss your conclusions about the most recent common ancestor of the pictured taxa | Given a variety of synapomorphies from different organisms, create a cladogram, identifying where the derived shared characteristics were acquired | Given a case study showing that a group of organisms have different relationships depending on the type of data used to construct the tree, use new information provided to evaluate the collective data and infer the best true relationship of the organisms |
Punnett squares and pedigree analyses | Given a Punnett square, identify components (genotypes or phenotypes; parents or offspring) of a given genetic cross | Given parental genotypes, make Punnett square to show or describe offspring's genotypes and phenotypes | Given parental genotypes in a word problem, student identifies variables and makes Punnett square to determine genotypic or phenotypic ratios of offspring. Student is provided with information regarding dominance, sex linkage, crossing-over, etc. | Given parental genotypes, make Punnett square to show or describe offspring's genotypes and phenotypes, and then solves a word problem with the new information. Student must infer relationships regarding dominance, sex linkage, crossing-over, etc. | Use pedigree analysis to develop a hypothesis for how a certain disease is transmitted | Weigh the relative value of different pieces of evidence (pedigree chart, incomplete transmission, linkage analysis, etc.) and determine the probability that an individual will develop a certain disease |
The BBT is not meant to be an absolute or definitive rubric; rather, the BBT is meant to be used as a general guide to aid both faculty and students in developing and identifying biology-related questions representing the different levels of Bloom's. As with all assessment methods, we expect the BBT to continue to evolve through an iterative process. Continuous feedback from students and faculty using the tool will inform its evolution.
DEVELOPMENT OF THE BLOOM'S-BASED LEARNING ACTIVITIES FOR STUDENTS
The BBT can also be used by students to help them identify the Bloom's level of exam questions that pose the greatest academic challenge. However, once these challenging areas have been identified, students also need guidance on how to modify their study habits to better prepare themselves to answer those types of questions. We therefore created the Bloom's-based Learning Activities for Students (BLASt; Table 3), a complementary student-directed tool designed to specifically strengthen study skills at each level of Bloom's. We determined which study activities provided students with the type of practice that would lead to success at each Bloom's level. For example, the first two levels of Bloom's rely heavily on memorization skills that can be reinforced by an individual student using flash cards and mnemonics. However, the remaining levels of Bloom's that represent HOCS are more readily achieved through both individual and group activities. The BLASt incorporates a range of study methods and can be used by students to refine their study skills to become more efficient and effective learners.
Bloom's level | Individual activities | Group activities |
---|---|---|
Knowledge (LOCS) |
|
|
Comprehension (LOCS) |
|
|
Application (LOCS/HOCS) |
|
|
Analysis (HOCS) |
|
|
Synthesis (HOCS) |
|
|
Evaluation (HOCS) |
|
|
IMPLEMENTATION OF THE BBT IN OUR CLASSROOMS
While developing the BBT, we found that the very process of developing the BBT was strongly influencing our own teaching in the classroom. The BBT was guiding us to ask and write better questions, develop more appropriate learning strategies, and assist our students in the development of their metacognitive skills. This tool provided us with a means to consistently apply the principles of Bloom's to biology concepts and skills, thus allowing us to better assess student-learning outcomes.
The following passages illustrate how we have applied the BBT at either a research-one institution or a liberal arts college in three different classroom contexts: (1) a small inquiry-based laboratory, (2) a large lecture, and (3) a medium-sized workshop setting. Table 4 presents the timelines of implementation of each teaching strategy. To facilitate a comparison of our different implementation strategies, we have compiled a chart outlining the strengths and challenges of each approach (Supplemental Material C).
Faculty use of the BBT in an undergraduate cell biology laboratory course | |
1st Quarter |
|
Postquarter |
|
2nd Quarter |
|
Postquarter |
|
Faculty and student use of the BBT in an undergraduate physiology course | |
Day 1 |
|
Day 2 |
|
Each day |
|
Prior to exam |
|
Exam |
|
Postexam |
|
Last day of class |
|
Student use of the BBT in biology workshops at a liberal arts college | |
Week 1 |
|
Week 2 |
|
Week 3 |
|
Weeks 6–10 |
|
Use of the BBT by a Faculty Member in a Laboratory Course at a Research-One Institution
In a small, upper-division, inquiry-driven cell biology laboratory class (two sections of 11 students each) at a research-one institution, the BBT was used to evaluate student performance and redesign course activities to enhance student learning. The class was taught during consecutive quarters with a new cohort of students each quarter. The primary writing assignment in the course (worth 1/3 of the total grade) was a National Institutes of Health (NIH)-style research proposal. This was a challenging assignment for the students as none had written a research proposal before this course and most (>75%) had no previous research experience. Over the course of the quarter, groups of three or four students read primary scientific literature on their topic of interest, formulated new hypotheses, and designed and performed a pilot study to gather preliminary data in support of their hypotheses (see Table 4 for timeline). Each student then communicated his/her ideas and findings in the form of a written research proposal in which the student posed a hypothesis and described a set of specific aims (i.e., specific research objectives for the proposed study, as defined in NIH grant proposal guidelines) designed to further test this hypothesis. The assignment also required students to provide expected outcomes of their proposed experiments and discuss possible alternate outcomes and limitations inherent in their research design. The assignment was designed to teach students how to synthesize their own data with existing data from the literature and to build a strong argument in support of a new hypothesis. Students turned in one section of the proposal each week (e.g., Background and Significance) and received written feedback. Common difficulties were discussed with the class as a whole; however, neither the grading criteria nor the rubric were made explicit to the students.
To facilitate evaluation of the students' research proposals, a grading rubric was developed (Walvoord and Anderson, 1998; Allen and Tanner, 2006). Students were scored from 1 to 4 for how well they fulfilled each of 12 criteria as well as for overall presentation (Table 5). Student performance was gauged both by looking at the percentage of students who earned full credit on a given criterion (Table 5) and also by determining the average percentage of possible points students earned for each criterion (data not shown). In reviewing these results, it appeared that certain criteria were much more challenging for students than other criteria. For example, whereas 41% of the students provided a well-thought-out and insightful discussion of their study's broader societal and scientific impact, <10% of the students were able to design specific aims that directly tested their hypothesis (Table 5). Others have assessed students' ability to write research proposals and identified similar areas of weakness (Kolikant et al., 2006).
Research proposal grading criteria1 | Percent of students fulfilling criterion2 | Level of Bloom's3 |
---|---|---|
Hypothesis and specific aims | ||
Context (logical development of hypothesis) | 50 | App/Anal |
Hypothesis | 33 | Synth/Eval |
Specific aims designed to test hypothesis | 9 | Synth/Eval |
Background & significance | ||
Logical introduction of background relevant to topic | 50 | Know/Comp |
Review of literature identifying gaps in knowledge | 27 | Synth/Eval |
Broader societal and scientific significance of study | 41 | Know/Comp |
Preliminary data | ||
Presentation of pilot study results | 28 | App/Anal |
Interpretation and relevance of pilot study | 28 | App/Anal |
Research design | ||
Overall design (appropriate methods, controls) | 32 | App/Anal |
Alternate outcomes for proposed study | 23 | App/Anal |
Limitations of proposed approach | 9 | Synth/Eval |
Methods | 32 | Know/Comp |
Presentation | ||
Overall organization, grammar, style, figures | 14 | None4 |
Subsequent to determining student proficiency in each area, the BBT was used to categorize each criterion based on the highest cognitive domain it demanded (Table 5). (Please note that the section entitled “broader societal and scientific significance” was ranked as knowledge/comprehension rather than application/analysis as the instructor had explicitly discussed the significance of this general area of research during lecture and students merely had to recall and focus the information for their specific study rather than apply knowledge to a new situation.) Not surprisingly, students performed best on criteria that required only a knowledge- or comprehension-level of thinking. Those criteria that demanded an ability to synthesize new ideas or critically evaluate a technique or body of knowledge proved to be the most challenging.
After assessing student performance on the research proposal and identifying the criteria that students found the most challenging, the instructor designed new course activities that would provide students with an opportunity to practice skills needed to complete this complex research assignment (i.e., better scaffold the assignment). Two major changes were implemented when the course was taught the subsequent quarter. First, the assessment methods were made more transparent by introducing students to the grading rubric at the beginning of the quarter. Students were also provided with numerical feedback, in addition to written feedback, on each of their drafts indicating how well they had fulfilled each of the grading criteria (e.g., on their hypothesis and specific aims section they might receive 3 out of 4 for developing a clear testable hypothesis, but only 2 out of 4 for designing specific research objectives that tested this hypothesis). Second, as suggested by the BLASt, students evaluated their peers' research proposals from the previous quarter. This activity served three purposes: (1) to further familiarize students with the grading criteria that would be used to assess their own proposals, (2) to build students' confidence by placing them in the position of evaluator, and (3) to provide students with student-created models of research proposals that they could use to guide development of their own proposals.
To assist students in applying the grading rubric to their peers' proposals, all students were asked to evaluate the same proposal from the previous quarter, and then a “norming session” was held in which the students received the instructor's ratings with further explanation as to why a particular numerical value had been assigned. Interestingly, students on average were harsher critics of their peers than the instructor in areas where they felt most confident (e.g., presentation style), whereas they awarded higher scores than the instructor in areas where they were less knowledgeable (e.g., research design). Students were then assigned a new set of three proposals that they evaluated individually. After reviewing the proposals, students convened in groups of four to act as a “review panel” to discuss the relative strengths and weaknesses of the three proposals and come to consensus on a rank order. These activities took a significant amount of class time, but ensured that students understood each of the criteria on which their own proposals would be scored at the end of the quarter.
Comparison of research proposal scores between the second and first quarter revealed some interesting trends. Criteria requiring the most complex thinking skills showed the most dramatic improvement (Figure 1). For example, the second quarters' students earned an average of 80% of the total possible points for discussing inherent limitations to their research design compared with only 61% in the previous quarter. Likewise, we observed a strong increase in student ability to interpret their data and design their own hypotheses, skills that require analysis and synthesis levels of Bloom's, respectively. As these data were derived from two different populations of students (fall and winter quarter), the students' scores were analyzed according to their rank order using a nonparametric Kruskal-Wallis test, which does not assume that the two data sets possess a normal distribution. Based on this analysis, all three of the most dramatic increases were found to be statistically significant (Figure 1).
Students' scores on criteria requiring LOCS did not show statistically significant differences between the two quarters, indicating that the two groups of students were equivalently matched in terms of their basal knowledge of cell biology. This lack of increase in areas of knowledge and comprehension also suggests that the newly incorporated activities primarily impacted students' HOCS. Students in the second quarter were less successful in describing experimental methods than their peers from the previous quarter; however, this is most likely attributed to the fact that students in the second quarter were asked to include methods that they were proposing to use (but had not used in the laboratory) whereas students in the first quarter were only required to include methods they had used to obtain their preliminary data (and were therefore very familiar with).
The large increases in student performance on some of the most challenging aspects of the assignment occurred after implementation of class activities designed to enhance HOCS. However, the gains in student achievement could also be attributable to unrelated factors including quarter-to-quarter variation in student motivation or differences in faculty performance. Future research will focus on distinguishing between these different possibilities.
As instructors, it is important that we recognize the complexity of the tasks that we are assigning students and prepare students appropriately for difficult tasks that require higher levels of thinking. As illustrated in this example, different sections of a research proposal require different cognitive skills. By recognizing which parts of an assignment are the most challenging, we can design specific activities or tools to help students succeed in those areas. Here, the faculty was able to use the BBT to identify areas in which students struggle and focus on improving the learning in these areas. The grading criteria were explicitly discussed and students were provided with structured opportunities to act as evaluators of other students' work. By sharing other students' work, it was possible to more clearly illustrate what “success” with a given criterion would or would not look like. These types of activities, based loosely on the cognitive apprenticeship model (Collins et al., 1991), may help prepare students for challenging assignments (Felzien and Cooper, 2005; Kolikant et al., 2006).
Faculty and Student Use of the BBT in an Undergraduate Physiology Course
Bloom's Taxonomy of cognitive domains was introduced during the second class period of a large (120 students) upper-division undergraduate physiology course at a research-one university. Introduction of Bloom's took only 15 minutes and focused on helping students learn the taxonomy and realize the potential it offered for enhancing their learning. To reinforce the concept, students were assigned the homework task of developing their own mnemonic for the levels of Bloom's (see Table 4 for timeline). For the first 10 minutes of the next class, a representative sample of mnemonics was presented, and students were asked to identify the strengths and weaknesses of each mnemonic. Before soliciting responses, the students were queried as to which level of Bloom's was required to complete these two tasks (i.e., creating a mnemonic and identifying the strengths and weaknesses of a mnemonic). In future classes, this activity would be referred to as “Blooming” the question.
Throughout the quarter, three to four questions on course content and concepts were asked during each class period, and the students were always asked to “Bloom” each question before answering it. “Blooming” in-class questions not only affords the students practice in using Bloom's with immediate feedback from the instructor but also allows the students to gain insight into which level of question they are having the most difficulty answering. This type of exercise strengthens student metacognition as it helps them monitor their mastery of the course concepts. Enhancing student metacognition has been found to be critical to student learning (Schraw, 1998; Bransford et al., 2000; Pintrich, 2002; D'Avanzo, 2003; Coutinho, 2007).
Physiology is a challenging subject for students as it is based on a mechanistic and analytical rather than descriptive understanding of organismal processes (Modell, 2007). As such, the discipline requires students to work predominantly at the higher levels of Bloom's Taxonomy. Few students enter the course prepared to use the HOCS required to succeed on exams; therefore, it is necessary to raise awareness of the challenge level of the exam before the exam is given. To this end, students were given a homework assignment of first categorizing each question on the previous year's exam according to Bloom's and then calculating the number of points on the exam associated with each Bloom's level. This exercise helped students gain an appreciation for the Bloom's distribution of the exam questions and allowed them to adjust their studying accordingly.
During the quarter the instructor used the BBT to categorize the Bloom's level of all exam questions. This allowed the instructor to compute a Bloom's distribution for each exam (i.e., 16% points at the knowledge level, 38% at the comprehension level, and 46% at the application level), which in turn indicated the cognitive challenge of the exam. Calculating the Bloom's distribution allowed the instructor to determine whether indeed the exam questions were aligned with the course content and learning goals. Postexam, in addition to the routine analysis of test performance (range, means, SD) the instructor also showed how the class performed at each Bloom's level. It was not surprising to find that on the first exam students earned 80% of the knowledge points, 70% of the comprehension points, and only 55% of the application-level points.
As the quarter progressed, the instructor recognized that it was important to provide students with their individual Bloom's scores. This was necessary as students frequently did not consider the class average to reflect their own performance, and though the Bloom's ranking of each exam question was included on the exam key, few students actually calculated their own Bloom's test score. Therefore, after the second exam was returned to the students, the students were instructed to enter their score for each exam question into an online data-collection tool. This data were then used to generate a Bloom's analysis of each student's test performance. The Bloom's test score is the percentage of points an individual student earns at each level of Bloom's (e.g., if they earned 10 of the 20 points assigned to application-level questions they earn a 50% application score). Students accessed their Bloom's test score through the grade-reporting portion of the course website. By this point in the quarter, the BLASt had been completed and made available to all students. However, students who earned <75% of the points at any Bloom's level were specifically directed to appropriate learning activities of the BLASt and strongly encouraged to incorporate those activities into their study and learning strategies. As individual Bloom's scores were not reported and the BLASt was not available until midway through the second half of the class, significant improvement in student performance on the second midterm was not anticipated.
Research on human learning has found that developing student's ability to monitor their own learning (i.e., metacognition) is crucial to successful learning (Schraw, 1998; Bransford et al., 2000; Pintrich, 2002; D'Avanzo, 2003; Coutinho, 2007). By “Blooming” in-class questions, students are provided with daily formative assessment of their learning while the Bloom's analysis of test performance provides the student with a more focused assessment of the type of question with which they struggle. The technique of providing students with a Bloom's test score in combination with recommendations for alternative learning methods from the BLASt gives students a simple and straightforward means to monitor and change their learning strategies in biology. Unfortunately, by the time the students received their personalized Bloom's analysis of their second test performance, only two weeks remained in the 10-week quarter, and there was not enough time for students to make meaningful changes to their existing study habits. As a result, it was not possible to show significant changes to student learning over the course of the quarter. In future quarters, the personalized Bloom's analysis of test performance will be introduced at the start of the quarter, and greater emphasis will be placed on devising methods to help students learn how to implement study skills appropriate for the academic challenge of the course.
After the quarter ended, students were asked what they thought about adding Bloom's to the course content. Below are two representative student responses:
I think Bloom gives students an increased insight into the different types of learning and application of knowledge that students do for a class, it makes explicit something that is maybe only understood at a subconscious level. I think it gives students more tools and increases the control they have when they are studying.
I remember initially thinking, “Why are we wasting valuable class time on Bloom's taxonomy?” I felt that Bloom's taxonomy was a burden, but I now use Bloom's taxonomy unconsciously to attack many problems. It is a method used to help organize my thoughts before I act.
Student Use of the BBT in Biology Workshops at a Liberal Arts College
Bloom's was used to promote pedagogical transparency and enhance students' abilities to design and answer questions in an upper-division interdisciplinary science program. Throughout the year-long program, students participated in weekly lectures, laboratories, seminars, and workshops cotaught by three different faculty who integrated topics in organic chemistry, biochemistry, cell biology, virology, and immunology. Workshops typically provided students with an opportunity to practice their problem-solving skills by answering faculty-generated questions in small groups.
The BBT was implemented in the immunology workshops. Thirty-six students received formal training in using the BBT, and then worked collaboratively in the subsequent 10 wk of the quarter to develop questions representing all different levels of Bloom's for a variety of assigned readings (Table 4). Students were first formally introduced to Bloom's in a half-hour lecture during which the faculty used biology sample questions to exemplify the different levels. After the lecture, small groups used the BBT to rank 45 biology and 20 organic chemistry questions from GRE subject tests and faculty exams. The faculty provided assistance throughout the activity, and students were required to submit their ranked questions for credit. This process allowed students to practice using the BBT for evaluating the different levels at which questions can be written and helped them to engage in discussion about the type of questions presented.
One wk after their initial training, students used the BBT to create questions from the content presented in eight primary literature papers that the students had previously read. Small groups of students were each assigned two papers for which they created two questions at each of the first five levels of Bloom's. The groups exchanged papers and associated questions, critiqued the level and design of the questions, and attempted to answer them. With faculty facilitation, each group presented their critique of and answer to one question to the entire class. The class then engaged in an open discussion about the material presented. These activities provided students with hands-on training for designing questions at different levels of Bloom's and set the stage for the remaining 8 wk of immunology workshops.
During week three, the faculty generated 10 questions at each level of Bloom's covering assigned reading in an immunology textbook. In their scheduled workshop time, students met in small groups to discuss and answer the questions. For homework students were required to individually answer and rank the questions according to Bloom's. Students received credit for both their answers to the questions and their completion of Bloom's rankings.
During the last 5 wk of the program, students were responsible for generating and answering their own questions based on assigned reading. Groups of five to seven students were responsible for writing a total of 20 weekly questions corresponding to the chapter that was being presented in lecture. Each week, a group generated four questions at each of the five levels of Bloom's. The night before the workshop, the questions were sent to the faculty and the best questions were selected and arranged in random order with respect to Bloom's ranking; the designated rankings were excluded from the final handout. In the workshop, authors of the questions served as peer teaching assistants while the other students worked to answer and rank questions. The authors were instructed to withhold the Bloom's ranking from the other students and to assist them only with finding the appropriate textbook material for answering the questions. Students were required to individually type up their answers and rank the questions according to Bloom's. These weekly assignments were turned into the faculty for grading, but students were only graded for their responses to the assigned questions and for completing the Bloom's ranking. Although exams and homework assignments given at The Evergreen State College are graded and scored, the college does not give cumulative numerical grades but rather narrative evaluations of a student's course work. This pedagogical philosophy enhances learning communities and provides an environment for effective group work. Students were held responsible for their participation in workshops by grading their individual responses to the questions.
The goals of the course activities were to teach students about Bloom's and let them practice using the BBT to rank and write good questions at different levels so that they could independently assess the level of their understanding of biology content in the future. Based on a show of hands in class, only one student had heard of Bloom's but did not feel as though they understood it enough to use it. While students were first practicing ranking questions, the instructor formatively assessed their knowledge of Bloom's and confirmed that none of the students in the course had any experience using it. However, by the end of the course, the students were very consistent in their independent ranking of the questions according to Bloom's. For 31 of the 51 questions, greater than 80% of the students agreed on the Bloom's ranking (Figure 2). This indicates that students who are trained to use the BBT are capable of writing and identifying questions at different levels of Bloom's. Students can apply this knowledge to their studying practices, evaluating the levels at which they understand concepts and adjusting their study skills to reach higher levels of Bloom's. These findings were highlighted by students in their final written evaluations of the program; some indicated that these exercises also helped them develop better questions about material they were learning in other areas of the program. The following are evaluation responses related to the use of Bloom's in the program:
Designing challenging questions proved to be often more difficult than answering them. Studying via question design is a skill that I will apply to new material in the future.
A huge part of this course was learning how to use Bloom's Taxonomy which is a ranking system for formal questions. Throughout the quarter groups were required to write questions as well as answer questions based on this ranking system. Learning Bloom's Taxonomy showed me how much effort goes into designing an exam or a homework assignment. I find myself wanting more.
All year long I engaged my peers in workshop and problem set collaboration, and while I always learn a significant amount in that setting, I was not comfortable with being led through a quarter's worth of assignments by students that knew less than me. However, I must add that [the faculty's] desire to instruct students in the art of thinking like a teacher and asking questions on many different levels of understanding was beneficial.
Learning the different levels of questions really helped me to take tests better and increased my capacity of grasping concepts.
Collectively, this suggests that formal training of students to use the BBT in ranking science questions, followed by substantive practice at writing and ranking questions at different levels of Bloom's Taxonomy, enhances their study skills and metacognitive development.
IMPLICATIONS FOR UNDERGRADUATE BIOLOGY EDUCATION
Assessment is the process of evaluating evidence of student learning with respect to specific learning goals. Assessment methods have been shown to greatly influence students' study habits (Entwistle and Entwistle, 1992). We agree with other educators who have argued that in the process of constructing a course, assessment is second only to establishing course learning goals for guiding course design (Wiggins and McTighe, 1998; Palomba and Banta, 1999; Pellegrino et al., 2001; Fink, 2003). Though many faculty establish learning goals for their courses, they often struggle with how to evaluate whether their formative and summative assessment methods truly gauge student success in achieving those goals.
Most faculty would agree that we should teach and test students for higher-cognitive skills. However, when faculty are given training in how to use Bloom's and practice ranking their own exam questions, they often realize that the majority of their test questions are at the lower levels of Bloom's. For example, at a national meeting for undergraduate biology education, 97% of the faculty who attended (n = 37) and received a formal lecture on using Bloom's to rank exam questions agreed that only 25% of their exam questions tested for higher-order cognitive skills (unpublished data). Therefore, most of the time we may not be testing or providing students with enough practice at using content and science process skills at higher cognitive levels, even though our goals are that they master the material at all levels. One explanation for this discrepancy may be that biology faculty have not been given the tools and guidelines that would help them to better align their teaching with assessments of student learning. To further emphasize this point, an analysis of exam questions from courses in medical school that should be aimed at developing HOCS (Whitcomb, 2006) are instead predominantly testing at lower cognitive levels (Zheng et al., 2008).
Developing strong assessment methods is a challenging task, and limited resources have been allocated to support faculty in this endeavor. Further, because of the current trend of increasing class size and decreasing teaching assistant support, multiple-choice exams are becoming the most practical assessment method. It is therefore increasingly important for faculty to invest the time necessary to create multiple-choice exam questions that test at the higher levels of Bloom's (Brady, 2005), as well as to develop integrative testing approaches such as requiring students to justify their answers of a small subset of multiple-choice questions (Udovic, 1996; Montepare, 2005). However, in order to accurately gauge student performance, we strongly encourage faculty to include short essay answer questions or other types of questions that test HOCS on their exams. This shift in assessment practice may require additional teaching support from departments and administrations, but we believe this is very important to the cognitive development of our students.
Our aim in developing the BBT was to make an assessment tool for use by biology faculty and students alike. To further facilitate this process, we have created a diverse array of biology-focused examples, inclusive of both specific skills (e.g., graphing) and subdiscipline content (e.g., physiology) that biology students typically encounter. These examples, in conjunction with the BBT, are designed to aid biologists in characterizing questions according to their relative cognitive challenge and, therefore, develop assessment methods that are more closely aligned with an instructor's learning goals. The BBT can also be used in conjunction with BLASt to help students self-diagnose their learning challenges and develop new strategies to strengthen their critical-thinking skills.
Our implementation of the BBT enhanced teaching and learning in a wide variety of instructional environments. Using the BBT, we were able to identify the cognitive levels of learning activities with which students struggle the most and adjust our teaching practices accordingly. The BBT also helped us to create pedagogical transparency and enhance student metacognition. As always, there is a trade-off when class time is used to develop metacognitive skills as opposed to focusing exclusively on course content. However, in our student-based implementation strategies of the BBT, Bloom's Taxonomy was fully integrated into the course subject matter (e.g., designing exam questions at different levels of Bloom's); anecdotal evidence from our students suggests that they continue to use Bloom's to guide their learning strategies in future classes. Given our experience and the well-documented importance of metacognition in student learning in all disciplines, including science (Schraw, 1998; Bransford et al., 2000; Pintrich, 2002; D'Avanzo, 2003; Coutinho, 2007), we consider the potential benefits students may gain from learning Bloom's to far outweigh any consequences of minimally decreasing course content.
We envision that the BBT could help faculty create biology questions at appropriate cognitive levels and in this way provide faculty with a means to (1) assess students' mastery of both biological content and skills and (2) better align their assessments and learning objectives. We believe that use of the BBT by both faculty and students will help students achieve a deeper understanding of the concepts and skills that are required to become successful biologists. On a broader scale, the BBT could aid in development of biology assessment tools that could then be used to examine levels of academic challenge between different types of standardized exams in the life sciences and to facilitate departmental and interinstitutional comparisons of college biology courses.
ACKNOWLEDGMENTS
We thank J. Dorman, S. Freeman, and M. Withers for critical review of the manuscript. We owe particular thanks to S. Freeman and his two undergraduate coauthors (Zheng et al., 2008) for providing us with the inspiration and the encouragement needed to pursue this work. A.J.C. would like to thank T.S. Gross for help with statistical analysis.