ALL VOLUMES SEARCH TIEE
Volume 6: Table of Contents TEACHING ISSUES AND EXPERIMENTS IN ECOLOGY
RESEARCH

Practitioner Research Improved My Students’ Understanding of Evolution by Natural Selection in an Introductory Biology Course

Introduction

“Teaching requires the recognition that education is ideological.”

— Paulo Freire (1998, Pedagogy of Freedom)

Large numbers of students that we face in our freshman biology classes have fundamental misunderstandings about evolutionary biology (Alters and Nelson 2002, Miller et al. 2006). Whether by design or by neglect (Brem et al. 2003, Griffith and Brem 2004), many pre-college school systems nationwide have graduated successive cohorts who harbor, express, and defend deeply held misconceptions not only about the basic processes of evolution (Munson 1994, Dagher and BouJaoude 1997, Alters and Nelson 2002, Anderson et al. 2002, Bowman 2008, NAS 2008), but also about the kind of thinking that led to discovering and constructing the evidence in support of evolution to begin with, i. e. their scientific epistemology (King and Kitchener 1994, 2004, Bransford et al. 2000, Lederman et al. 2002, Sinatra et al. 2003, Wills 2004, Smith and Wenk 2006).

One widely used response has been to inventory students’ misconceptions in evolutionary biology and design curricular activities that confront these misconceptions directly (e. g., Treagust 1986, Bishop and Anderson 1990, Rutledge and Warden 1999, Anderson et al. 2002, Passmore and Stewart 2002, Hokayem and Boujaoude 2008, Libarkin 2008). However, as Wandersee et al. (1994) caution, simply telling students that their “mental model” of a phenomenon is incorrect is ineffective at displacing their misconceptions. Following an extensive review and synthesis of evidence about the roles of students’ alternative conceptions and misconceptions in their understanding and acceptance of evolution, Alters and Nelson (2002) advocate that a “key improvement” in teaching evolution would be to use more active learning and constructivist approaches in our classroom pedagogy.

The challenge then becomes “how?” Thus, how might one start with somewhat pre-formatted tools (e. g., Anderson et al. 2002) or simple writing assignments (e. g., Hein 1999) to identify students’ misconceptions not only to the instructor, but also to make these known to the students and engage them in the process of displacing their own misconceptions so that the “expert” constructs may be heard, accommodated, and assimilated (Wandersee et al. 1994)? These are classroom research questions, and faculty who will be successful at this must become truly scientific in their approach to teaching. For example, if students perform poorly on a quiz or exam, this should be seen as a “discrepant event” (Liem 1981), and the challenge is to avoid “deficit explanations” for poor performance (e. g., the students are unintelligent or don’t work hard enough, Elmersky & Tobin 2005), and instead rise to engage in “scientific teaching” (Handelsman et al. 2004) to discover how students arrived at their observed misunderstandings and what works to effect the acquisition of both expert knowledge and expert ways of knowing (Gardner 2004). Knowing the reasons for students’ misunderstandings derived from, or despite, our teaching allows us to revise and improve our curriculum, instruction, and assessment and cause improved student learning in science (NRC 1996, Bransford et al. 2000, D’Avanzo 2003, D’Avanzo et al. 2006, Morris et al. 2007). Since these research activities are essential to improving teaching practice, this activity has been referred to as teacher “practitioner research” (Stenhouse 1975, Jarvis 1999, Allwright 2005, Stringer 2008). The same thinking and model of faculty development needs to be applied to undergraduate faculty who teach evolution (Grant 2008).

Since 2000, I have been engaged in a long-term practitioner research project in my fall freshman introductory course in evolutionary biology at Widener University. I present evidence that my methods of practitioner research over the past 8 years, and in particular a major course revision including a novel method of confronting students’ misconceptions that I developed in 2006, have contributed to significant reductions in my students’ misconceptions and significant increases in their demonstrated understanding of basic concepts of evolutionary biology.

[ TOP ]

Methods

In the fall of 2000, following a complete revision of the freshman and sophomore majors’ curriculum in Biology at Widener University, Robert Morris and I created the fall freshman course, “Biology 161: Evolutionary Ecology.” The course is divided into thirds: (1) Origins (basic review of the origin of the Universe, our solar system, pre-biotic chemistry, and life on Earth), (2) Genetics (micro- to macroevolution, and the evolutionary diversification and classification of life), and (3) Evolutionary Ecology (individuals, populations, communities, and ecosystems) and Conservation Biology. I taught the first and last third of this course each fall from 2000-2007 with 45-60 students each year. Evolution and coevolution are central concepts to this course and these terms appear prominently in eleven and five out of fifteen weeks or the course, respectively (see Supplemental Figure S1 for the basic lecture outline for this course as well as the numerous places where the term evolution are coevolution are featured). In lecture, I frequently made use of active-learning pedagogical techniques including Turn-To-Your Neighbor, Minute Papers, and Guided Discussions (see TIEE Teaching Resources).

Assessment of Concept Acquisition

This paper reports multiyear results (2000-2007) from two sets of quiz/exam questions that serve to elicit students’ understanding of (1) the definition of evolution and (2) the process of evolution by natural selection.

(1) Definition of Evolution. I put the question “Please offer a brief and concise definition of Evolution” on the Final Exam each December from 2000 to the present. I re-read and re-scored all Final Exam responses (n=379) to this question using the rubric in Figure 1, which I also used to compile frequencies of students’ misconceptions.

According to the rubric (Figure 1), students must have noted the role of “genetic change” in evolution to excel in this question (the emphasis is on microevolution). Next, failing but score-able responses exhibited major misconceptions such as equating evolution with natural selection, or implying teleological directed evolution based on vaguely adaptive need. Also, a student’s performance at simply writing clear sentences affected their score (max of minus one point).

Figure 1. Scoring Rubric (max +10 pts) for the question “Please offer a brief and concise definition of evolution.”

Since the beginning of 2005, in addition to final exams, I administered this question as a pre-test and on several quizzes, and I also saved copies of all pre-tests, quizzes, and mid-term tests. Thus for 2005 through 2007, I have an additional data set of responses (n = 647) to indicate changes in students’ performance on this question pre-, during, and post-instruction. I re-read and re-scored all of these responses for this paper for consistency.

(2) Evolution by Natural Selection. One of the most useful instruments I have to measure students’ conceptions about evolution by natural selection is the “Dino Neck” question (Figure 2). I administered this survey to all of my freshman students during the first and last weeks of class from 2000 to 2007.

Figure 2. “Evolution Made Simple” assessment question from Diane Ebert-May (2000)

To excel, students must have noted for both the tree and then for the animal that (a) coexisting individuals in the tree/animal populations differ from each other (in tree height or neck length) due to random genetic variation, (b) stated that due to these differences trees/animals differ in ecological performance (survival and reproduction, or fitness) since taller trees are eaten less/ longer necked animals get more food, which is natural selection, and (c) since these taller/ longer necked individuals leave more offspring that would inherit these traits, the populations of trees/ animals subsequently increase in their proportions of taller/ longer necked individuals – which is evolution. Lower scores stem from major misconceptions such as stating that variation among individuals and/or over time is directed by teleological need, or that variation is mere growth. Also, a student’s writing performance slightly affected the score. The complete scoring rubric for this question is available in Grant (2008) and I also used this rubric to compile frequencies of students’ misconceptions. I re-read and re-scored all of these responses for this paper for consistency.

Importantly, and in contrast to the “definition of evolution” assessment for which I used the same exact question repeatedly on tests and quizzes, my students only saw the “Dino Neck” question for the pre-course and final exam post-tests. Instead, in class and on quizzes I made up new questions modeled after the “Dino Neck” question that I scored using the same rubric but with different content. These new questions assessed students’ understanding of evolution and coevolution for predators and prey, competitive character displacement, and mutualism.

Course Revisions in 2005 and 2006

Beginning in fall 2005, I included almost daily 2-5 minute short-answer surveys of students’ pre-conceptions about key areas of content before these were discussed in class, although the syllabus remained the same as in previous years. However, it was apparent after the fall 2005 semester that student learning of the basic concepts of evolutionary biology was somewhat improved, but still inadequate. Many students who presented evidence on pre-tests that they harbored substantial misconceptions in fact remained highly resistant to instruction, and often defended their misconceptions using course appropriate terminology, but incorrectly, on the course final exam. In other words, many had hijacked course content in service of their misconceptions.

Beginning in fall 2006, I literally turned the tables regarding what I did with the pre-test data. Instead of my just reading and synthesizing this information and using it to measure the summative effects of my teaching in a positivist paradigm, I designed a plan to directly confront my students with their misconceptions and engage them in the shared struggle to dislodge these misconceptions so that expert learning could occur. Specifically, for each pre-unit survey, I presented histograms of their survey responses (prior knowledge and misconceptions) at the beginning of the next class. During this time, I specifically addressed the major categories of students’ prior correct knowledge and major misconceptions (why such and such an idea was “wrong” and why a different concept was “right”), I re-framed the outline of the class conspicuously around and sequentially addressing these major misconceptions, and I re-projected the misconceptions slide multiple times on multiple days.

For example, Figure 3 shows the histogram of students’ responses on the “definition of evolution” question from the pre-test assessment in September 2006. I showed this histogram multiple times in class, and these misconception categories formed the bases to the lecture content on evolution throughout the semester (and a similar graph from fall 2007).

Figure 3. Histogram of the frequencies of students’ responses on the “definition of evolution” question from the pre-test assessment in September 2006. This is the histogram I projected multiple times in class and these misconception categories formed the bases to the lecture content on evolution.

For a specific example, in class I referred to the histogram showing their misconception that evolution and natural selection are synonymous while I presented how other factors can cause evolution such as mutation, gene flow (migration), genetic drift in small populations (founder effects, genetic bottlenecks, etc). Next, returning to natural selection, I presented teleological views such as by Lamarck and showed why these might seem reasonable, but because they lack mechanisms these are all unfounded. Lastly, regarding the misconception that evolution by natural selection only occurs over long times, I presented evidence from the famous study by Peter and Rosemary Grant (chronicled in Jacob Weiner’s [1994] The Beak of the Finch, the first three chapters of which are required reading) showing that populations of Darwin’s finches can evolve rapidly from one generation to the next under selection. As I mentioned before, these are the same topics that I “covered” in previous years, but the difference in 2006 and 2007 was that these topics were presented in direct confrontation with students’ misconceptions.

In addition, I asked them in guided discussions and turn-to-your neighbor activities to visualize and reflect upon the kinds of evidence and arguments I needed to present in class, and the kinds of things they needed to do in and out of class, that would help them to see the difference between the expert knowledge and ways of knowing I wanted them to attain versus those exhibited in the pre-tests. Thus, my methodological research paradigm for 2006 and 2007 moved away from viewing my students as research objects and toward a model of their engagement in their learning in a participatory action research, or post-positivist, mode (McTaggart 1997, Hollingsworth 1997, McNiff 2002, Reason and Bradbury 2006, Stringer 2008). In many instances, this had little effect on the span of content already slated for any particular class from the syllabus, but simply involved rearranging the sequence and changing the emphasis of this or that idea.

A second major change in the course in fall 2006 was a rather substantial content reduction and shifting of emphasis, which was mandated in part by the extra class time that needed to be devoted to increased writing and discourse-based instruction. I removed the “Origin of Life” story from the first third, and I moved the first half of the last third on “Individual and Population Ecology” into the beginning third. The major content areas were largely unaffected. For example, basic topics in individual design (energy processing – basic biochemistry of metabolism and photosynthesis) were simply presented in the context of environmental adaptation and physiological ecology (further discussion of these changes is available in Grant (2008) and a summary graphic is in Supplemental Figure S1).

To reiterate, in my view the key innovation in 2006-2007 was to directly confront and make use of the students’ misconceptions as the bases to new outlines of topics that I then covered in class. In other words, beginning in 2006, I re-cast my whole in-class lecture pedagogy to teach directly to my students’ misconceptions about the content of my course. Nonetheless, it also was clearly a good idea to reduce the sheer volume of material I asked my students to learn and be able to manipulate.

[ TOP ]

Results

(1) Definition of Evolution

Between 2000 and 2005, the fraction of students correctly providing a brief and concise definition of evolution on the final exam (on a 10 point scale, see rubric in Figure 1) between 2000 and 2005 was only about 50% (Figure 4). This low percentage was one of the main factors motivating my course revisions for fall 2006 (see Methods). Beginning the year of my course revisions in fall 2006, there was a dramatic increase in this fraction (exceeding 90%), which of course also was associated with dramatic decreases in frequencies of students’ misconceptions for 2006 (all of which dropped below 5%). According to an analysis of variance on the numerical scores for this question, there was a significant increase in students’ scores for 2006 and 2007 relative to other years (SAS Proc GLM, Duncan’s Multiple Range Test, P<0.05).

Figure 4. Students' scores on the final exams (2000-2007) on the “definition of evolution” assessment question. Also included are the major categories of misconceptions exhibited (see Figure 3).

In all three years for which I have pre-post data, the fraction of students exhibiting an understanding of the meaning of the word “evolution” as a result of taking my course increased dramatically (Table 1, Figure 5). In addition, the fraction of students with particular misconceptions decreased by the end of the semester (Figure 5). In 2006 and 2007, students improved much more rapidly earlier in the semester than they did in 2005, with sharp declines in frequencies of all misconceptions by the end of the first unit.

Table 1. Pair-wise t-tests on students’ scores on the pre-test vs. final exam for fall 2005, 2006, and 2007 on the “definition of evolution” assessment question.
semester pre-test final exam t-stat d.f. P
fall 2005 3.78 7.84 10.20 44 <0.001
fall 2006 4.04 9.51 21.02 50 <0.001
fall 2007 3.99 8.79 18.56 66 <0.001

Interestingly, in 2006, a sharp drop occurred between “Exam1 Oct06” and “Quiz7 Nov06” between which many students apparently slipped out of calibration during the 4 week unit on genetics. For fall 2007, with this knowledge from fall 2006, we made adjustments in the course (more reviewing) to enhance student retention. As a result, in fall 2007, there was a much less noticeable dip in performance at this same stage. In addition, individual student’s trajectories reveal tremendous variation in performance from one assessment event to the next (see Supplemental Figure S2).

Figure 5. Students’ scores on the pre-test, quizzes, and exams for (A) fall 2005, (B) fall 2006, and (C) fall 2007 on the “definition of evolution” assessment question. Also included are the major categories of misconceptions exhibited (see Figure 3).

(2) Evolution by Natural Selection: the “Dino Neck” question.

For 2000-2005, very few of the students scored 8, 9, or 10 (about 3%) on the “Dino Neck” question (Figure 6); thus few understood how to answer this question even after considerable instruction on numerous occasions about the process of evolution by natural selection. In fact, most students (about 85%) exhibited major misconceptions, such as that phenotypic variation in the tree and animal stem from directed evolution based on Lamarckian or teleological need. In contrast, the results for 2006-2007 show a huge improvement in the fraction of students who scored 8, 9, or 10 (about 54%). Thus, more than half of my students correctly answered this question following course revision, and the average scores were significantly higher as well (for 2000-2005, average = 4.38, n = 265, and for 2006-2007, average = 7.36, n = 118; t-test unequal variances = 12.70, P < 0.001).

Figure 6. Simplified “Dino Neck” question scoring rubric on the left (see Grant 2008 for complete the rubric), and frequencies of students’ scores on the final exams for 2000-2005 and 2006-2007.

When considering students’ pre-post scores on the “Dino Neck” question, pre- and post- scores were all very similar and abysmally low (averaging between 4 and 5) for 2000 to 2005 (Figure 7), which stemmed from major misconceptions that my course did not displace (see Figure 3). However, for all pre-post data for 2000-2005, the post-test scores were significantly but only slightly higher than the pre-tests (4.01 vs. 4.40, pairwise t-test = 3.74, P <0.001, n = 255). Yet, both averages were between 4 and 4.5, which means that many of my students entered and exited my course maintaining deeply non-Darwinian misconceptions that were inert to instruction in these years. In contrast, in 2006 and 2007 post-test scores on the final exam were significantly and greatly higher than pre-test scores (3.56 vs. 7.34, pair wise t-stat = 16.71, P<0.0001, n = 115).

Figure 7. Averages of students’ scores on the pre- post-tests on the final exam “Dino Neck” question for 2000 to 2007, n = 370).

When one compares students’ “Dino Neck” pre-post scores, one can see the dramatic effect of course revision on most students’ understanding of evolution at the individual level. Figure 8 shows the counts of the number of students with a given “Dino Neck” score at the beginning of the course (columns) versus the same student’s score at the end of the course (rows) for the period 2000-2005 (left) and 2006-2007 (right). In other words, each cell shows the count of students by score pre-course in September versus their post-course score in December.

For 2000-2005 (Figure 8), although there are about twice as many students above (n=118) than below (n=64) the diagonal (i. e., more improved than not), only 15% of post-tests scored above 7 at the end of the course, which is really the minimum for Darwinian thinking. In contrast, for 2006-2007(Figure 8), 90% of the students scored above the diagonal post-test (% = 105/115) whereas only 3% scored worse on the post-test. Thus, under the revised course design, 90% of my students improved in their ability to demonstrate an understanding of evolution by natural selection with over 50% scoring 8, 9, or 10. In addition, there were significant pre-post decreases for every major category of misconception (see Figure 3) and increases in every major facet of evolutionary understanding (e. g., fewer students invoked teleological explanations, and more students implicated heritable variation among coexisting individuals and the important role played by random genetic mutation in creating variation), the details of which are reported in Grant (2008).

Figure 8. Counts for students’ September Dino Neck score versus the same student’s December Dino Neck score for 2000 – 2005 (left) and 2006 – 2007 (right).

Comparing students’ responses at the end of the course on the “Dino Neck” and “definition of evolution” questions reveals an important disconnect in learning that was significantly mitigated by course redesign. Figure 9 shows the counts of students by score for the “definition of evolution” question (columns) vs. counts by score for the “Dino Neck” question (rows). In 2000-2005, note that a large number of students who excelled at defining evolution (scoring 8, 9, or 10) showed little or no ability to apply this definition to the conceptual application in the “Dino Neck” question (about 50% lie in the “disconnect” circle in Figure 9). In fact, students’ pairwise responses were uncorrelated (R2 = 0.012, n = 247, P > 0.05). This suggests that another important layer of latent misconceptions is held by my students about how to apply knowledge across cognitive domains, i.e., the application of factual knowledge to challenges of knowledge synthesis.

However after the course revision in 2006-2007, there was a significant decrease in the number of students exhibiting this cognitive disconnect (Figure 9) (Fisher’s 2*2 Exact test comparing totals 2000-2005 vs. 2006-2007 for values within both x>8 and y<7, P<0.001). Indeed, most of the students performed well on both questions (scores of 8, 9, or 10) in contrast to the dispersion of scores in the 2000-2005 era. Thus, the combination of course revisions also affected an important outcome in student learning across cognitive domains in 2006 – 2007.Students were better able to apply factual knowledge about evolution to the challenges of knowledge synthesis in a narrative explanation of the process of evolution by natural selection. Devising teaching methods that enable students to make this type of connection across cognitive domains (as in Bloom’s Taxonomy) pose another layer of important research questions for this project.

Figure 9

Figure 9. Comparison of final exam “Dino Neck” vs. “definition of Evolution” scores for all students in 2000-2005 (left, n = 247) and 2006-2007 (right, n = 118).

[ TOP ]

Discussion

The suite of course revisions I instituted in fall 2006 significantly decreased students’ misconceptions and improved their learning about evolution and the process of evolution by natural selection. The revisions included modification of course content and reduction and radical redesign of the presentation modality of this course content. In particular, I used student pre-test responses to estimate class-wide frequencies of misconceptions about evolution and the process of evolution by natural selection. I then presented frequency histograms of their misconceptions directly to them in class, engaged them in reflective discourse to address these misconceptions, and then I rearranged and presented course content to systematically confront, address, and displace these misconceptions as “expert” knowledge and ways of knowing were transmitted. Thus, the students’ demonstrated misconceptions drove the course outline and methods of content delivery.

The major types of misconceptions decreased in students’ post-test responses following instruction in the revised modality. For example, more students recognized the importance and nature of phenotypic variation as occurring among co-existing individuals, more students understood the role of random mutation as the origin of these within population variants, more students appreciated the action of natural selection as selecting among these coexisting variants, and more students jettisoned teleological/ Lamarckian explanations for evolutionary change based on need. Finally, more students demonstrated improved cognitive capacity at applying factual knowledge across cognitive domains to synthesize information in the context of an evolutionary narrative. In sum, more of them got it and I hypothesize that many may have done so because they were more actively engaged in the research process into their own learning.

As Howard Gardner (2004) so eloquently relayed, “It is my belief that, until recently, those of us involved in education have not appreciated the strength of the initial conceptions, stereotypes, and ‘scripts’ that students bring to their school learning nor the difficulty of refashioning or eradicating them. (p. 5)” The challenge of identifying students’ misconceptions and designing pedagogies to dislodge them is an important design principle of constructivist classrooms (Fox 2001). A very large literature base exists applying these principles to address students’ misconceptions about ecology and evolution (Munson 1994, Dagher and BouJaoude 1997, Alters and Nelson 2002, Anderson et al. 2002, Bowman 2008, Hokayem and Boujaoude 2008, and references therein). Upon examination, all of the misconceptions categories I have detected as prevalent in my class have been previously identified in the literature as being deeply resistant to instruction. However, I know of no other studies that have undertaken as intensive a program of systematic long-term practitioner research on evolution as I report here, nor one that uses misconceptions inventories to drive the curriculum as I describe here.

This project underscores the value of long-term practitioner research in two major ways. First, data from pre-post tests in 2000-2005 showed that important concepts in evolutionary biology were not being learned, and in spite of repeated instruction, I was not displacing their easily identifiable and deeply entrenched misconceptions about evolution. These data both mandated and molded the course revision. Secondly, the practitioner research methodology provides guidance away from a positivist research paradigm and more towards a program of reflection and engagement (Stenhouse 1975, Jarvis 1999, Allwright 2005, Stringer 2008). Whereas positivism had very successfully led to identifying exactly why my curricula and instructional methods were less effective than I desired, my practitioner research paradigm “as a way of knowing” (Grant 2008) unveiled a pedagogy for my course towards a modality of confrontation of misconceptions, content reduction, and reflective learning by my students that was both effective and potentially transformative (Cochran-Smith and Lytle 1999). In sum, I brought their data directly to them approaching an “action research” model to enlist and engage them in reflective problem solving in their own learning (Wandersee et al. 1994, Hollingsworth 1997, Allwright 2005).

[ TOP ]

Practitioner Reflections

Over the past eight years while researching my teaching and student learning in this course, I have developed a “personal theory of practice” that is parallel to, yet very different from, the kind of scientific methodology that I was trained to use for knowledge discovery in my own education as a scientist (Carr 2006, Grant 2008). I summarize my new understanding in Figure 10 below. Practitioner research is deeply rooted in the Aristotelian concept of Praxis (Carr 2006), and represents an entirely different “way of knowing” about the research cycle of improving teaching and learning than is provided by traditional positivist scientific research (Grant 2008).

Figure 10. Research Cycle of Practice (Kolb 1984, Grant 2008). This cycle begins with observations of one’s classroom practice from which one determines the effects of one’s teaching on students’ learning, and one reflects upon and inductively hypothesizes causal relationships toward a “Personal Theory of Practice” (Carr 2006), according to which students are hypothesized to be learning (or not) due to one’s teaching (curriculum and instruction).  Then, one can make revisions to one’s teaching, return to the classroom, and repeat the cycle again — hopefully this time with improved learning outcomes. Practitioner research has a very different primary goal than does traditional scientific research. As summarized by Stringer (2008), practitioner researchers “engage in careful, diligent inquiry, not for purposes of discovering new facts or revising accepted laws or theories, but to acquire information having practical application to the solution of specific problems related to their work.”

Among the most important outcomes of this project is that I have learned several key analytical approaches that have equipped me with important and immediate positive responses to when my students do not perform up to my hopes and expectations. Instead of resorting to deficit explanations (Elmersky & Tobin 2005) when students fail, blaming either them, me, others around me, or all of the above, I now have a clear set of evidence-rich alternative directions in which to focus my efforts and design the next round of course revisions. Grading quizzes and exams for example has become much more of an intriguing data collection activity and much less of a mind-numbing and depressing drudgery. I have regained a lot of the thrill of discovery learning that got me into this profession to begin with.

Many however would counter that it’s all well and good for me to feel better about my teaching, but consequences matter the most – what is the evidence that my students are learning evolutionary ecology better?

The evidence I presented to address this question comes from my efforts to teach the meaning of the term “evolution” and the process of “evolution by natural selection.” Results indicate that when I added specific types of classroom instruction which I call “reflective constructivism” in fall 2006 as part of the TIEE “practitioner research” program, there were reduced frequencies of major misconceptions and significant improvements in students’ learning of the content and process of evolution by natural selection. Although I am not at this point able to statistically partition the gains in learning to specific changes in teaching, the main point is that gains occurred because of the cluster of changes arrived at through my practitioner research -- these were not random changes, but arose through analysis of performance, reflection, and the development of my personal theory of practice, which is the cycle of practitioner research (Figure 10).

At this point, I hypothesize that the approach of confronting student misconceptions was effective for several reasons. For one thing, I believe that my students perceived at least a partial transfer of ownership of the traditional lecture material from my domain, where I determined what was presented how and when, to a modality in which their attributes (prior knowledge, misconceptions, and struggles to learn new content and concepts) were elevated in prominence. The message was clear that what they knew and didn’t know mattered to me and greatly affected what we “covered,” when we covered it, and how we covered it in class.

Perhaps it is because this is a very narcissistic generation that lights up when they hear talk about themselves. But, I enjoin more likely it is because most of my students may have yet to really learn how to be active listeners and participants in a lecture-based classroom, and thus many feel lost as soon as they walk in the room. But when I projected a histogram of misconceptions (Figure 3) and I talked about them to them in class, they would all be looking at me, listening intently, focused upon my observations about them.With their reflections on their pre-tests in hand, they were engaged in the challenge of figuring out what needed to happen in and out of class for each person to improve in the course. Thus, I offer that a major reason why student performance might have increased was because the course revisions made the learning process more personal, relevant, and accessible to them – their struggle with learning this material was really about their struggle with learning about themselves (Schön 1983, Boud et al. 1985, Angelo and Cross 1988, Samuels & Betts 2007, Stringer 2008).

Nonetheless, I am still at best only seeing 60-80% of my students demonstrate understanding of these fundamental issues in evolutionary biology. Clearly, I have a long way to go, and I am not alone. This need for teaching teachers how to conduct practitioner research was articulated in the USA more than a decade ago in the National Science Education Standards in which a key facet of teacher professional development was to change teachers from “consumers of knowledge about teaching” to “producers of knowledge about teaching” (NRC 1996).

Future Directions:

  1. My principal next steps involve statistically partitioning the gains in my students’ learning to specific changes in my teaching. Given the substantial variation in performance, both among students and within the same student on different dates (e.g., Supplemental Figure S2), I doubt that one method will be the panacea for all; nonetheless, a small number of devices are likely to be effective for most. To help my students learn the complex concepts of evolution, my plan is to improve my students’ metacognitive processing abilities by having them reflect upon their own learning struggles to displace inaccurate misconceptions they harbor, not only about the course content, but more importantly about how they learn. What are the in-class activities, readings, study habits, peer-networks, and assignments that make them more self aware and elicit recognition of their own misconceptions in their prior knowledge and ways of knowing so that they can accomplish the tasks I set before them? I need each of them to adopt a practitioner research modality as in Figure 10 and develop personal learning portfolios that document their own personal theories of learning. My hope then is to determine the spectrum of effective pedagogies from these portfolios using exploratory statistics.
  2. I need to make greater use of published misconceptions inventories (e.g. Anderson et al. 2002) and analysis methods (e. g. ethnographic analysis, Tobin et al. 2001) to improve my identification and classification of the misconceptions my students harbor as well as enable my students to self diagnose through online assessments and course discussion forums. How do I use misconceptions pre-tests to identify students at risk (or who are advanced) and design interventions to improve their success and retention?
  3. I need to make greater use of multivariate statistics to cluster “errors” and identify underlying “latent misconceptions” not only to reduce the pure number of issues to address, but also to adopt more of a systems approach to designing misconceptions interventions.
  4. I need to explore experimentally, by making use of the different sections of the same course, what methods or combination of methods best displaces students’ diverse ecology misconceptions beyond just those related to evolution.
  5. The pattern of temporal variation in individual students’ scores on the “definition of Evolution” question in Supplemental Figure S2 is both astonishing and alarming. Why do students exhibit such dynamic temporal variation in performance on the same question over such a short time? How can I improve my use of class time and design better assignments to decrease this variation and improve all students’ scores?
  6. I wish to improve the scoring rubrics and measure the reliability of their use. As I relayed above, I rescored all “≡Evolution” and “Dino Neck” responses for this paper, and although I have careful notes, complete archives, and I went to great lengths to maintain calibration over the duration of this effort, there will always be a concern about single-rater reliability. I need to measure this effect.
  7. How do I pursue the next logical step in “practitioner research” to engage my students in genuine “action research” in their own learning community (Hollingsworth 1997, Elmesky & Tobin 2005)? I would model such a program on published advances in freshman learning communities and theories of social constructivism, and teach my students to research their own learning styles, processes, and outcomes in parallel to my standard measures of course performance and my model of learning in Figure 10. My bet is that the social dimension will accelerate metacognitive development and thereby result in increases in academic performance, agency, and self-efficacy.

Collaborators on any of the above topics are welcome! Please email me at bwgrant@widener.edu to get involved.

[ TOP ]

Supplemental Figures

Supplemental Figure S1. A) Bio161 lecture outline for 2000 to 2005 showing the weeks during which evolution and coevolution were prominent concepts.B) Bio161 lecture outline changes illustrating the redeployment of major topics and removal of technical content on the “Origin of Life”.

Supplemental Figure S2. Students’ individual scores on the pre-test, quizzes, and exams for (A) fall 2006, and (B) fall 2007 on the “definition of evolution” assessment question. The trajectory of the averages are in green. These data reveal a very important point about student performance on these kinds of assessments — individuals exhibit tremendous variation in performance from one assessment event to the next, and the vast majority of students show paths that bear little resemblance to the class averages. Most people are all over the place, ostensibly winking in and out of understanding. Learning is a highly non-linear process.

[ TOP ]

References

 [ TOP ]