Evaluation & Research on the Effectiveness of TIEE

April 2006 Update


Evaluation and research on the effectiveness of TIEE are essential so that we better understand aspects that are valuable, and what features need adjustment. Over the last several years Deborah Morris and Dan Udovic have headed up the evaluation of TIEE. We include here a summary of findings reported in D'Avanzo et al. 2006.


TIEE Users

For the TIEE study, the evaluators used surveys and interviews. The surveys provided descriptive data about a range of participants, while interviews allowed more in-depth study of some participants' views. The majority of survey respondents (61%; N=59) teach at liberal arts colleges, 26% are from research universities, and the rest are from community colleges and other institutions. The large number of liberal arts faculty was expected because teaching is emphasized at these schools, but we were pleased that a quarter of respondents taught at universities with large student populations. Over half of undergraduates are enrolled in community colleges and few faculty at these schools participated in the survey. We have submitted a proposal to NSF that includes plans for increasing community college faculty engagement with TIEE.


Role of TIEE in Teaching

About 50% of survey respondents rated themselves as "very familiar" with non-traditional pedagogy. Despite this, respondents allocate 25% or less of their course time for active learning, indicating that even faculty informed about active teaching primarily rely on traditional approaches in their classrooms. Ninety-three percent of those surveyed agreed or strongly agreed that TIEE had helped them improve their teaching of ecology. More specifically, respondents indicated that they used more active learning strategies and inquiry-oriented teaching approaches. However, while cooperative learning or inquiry approaches were used by many, relatively few faculty had employed alternatives to traditional tests or formative (ongoing) evaluation.

Adaptation and innovation of TIEE
One focus of our evaluation was the degree to which faculty modify activities and ideas to match their own teaching styles and the needs of their students. A key finding from both the survey and interviews was that faculty who use TIEE often adapt or modify the materials. Only 20% surveyed said that they used the TIEE materials "as is," while the rest used selected elements of TIEE activities or adapted them for a different use.

High quality ecological and pedagogical material
Users state that they value TIEE because of the high scientific and pedagogical quality of TIEE materials. Survey respondents indicated overwhelmingly that TIEE materials were scientifically accurate (100% "agreed," or "strongly agreed") and focused on core ecological principles (98%). In interviews, users often used the term "rigorous" to refer to both the scientific and pedagogical aspects of TIEE and that the quality far exceeds the type of labs generally published on the web or in lab manuals. Users frequently noted that TIEE fills a major need of helping students learn ecology through the use of real-world data and investigations.

Creating TIEE: sharing and scholarship
A major goal of TIEE is to provide a peer-reviewed venue for ecology educators to publish their work in a way that will be recognized in their institution's promotion and tenure process. Over 60% of individuals familiar with TIEE believe that publishing in TIEE would carry at least moderate weight in their institution's faculty evaluation processes.


Scientific Teaching with TIEE

The evaluation showed that few faculty were methodically studying the impact of their teaching on student learning. To address this, we are working closely (e.g., via workshops and conference calls) with a team of 15 selected faculty (see list below) from a range of institutions who are systematically studying possible impacts of TIEE on their teaching. These teams will present five posters at the 2006 ESA meeting in Memphis (http://www.esa.org/memphis/) describing their findings and effects of doing this research on their teaching.

These faculty have identified measurable outcomes — such as students' ability to make figures from raw data — and are using a variety of approaches, including pre/post tests and surveys, in their studies. Replicate measurements in the same course over a semester and several years plus across institutions can potentially result in publishable findings.

Our hypothesis is that these "practitioner researchers" will have a deeper understanding of why the student-active approaches featured in TIEE (such as groupwork) promote increased student learning and will be more committed to their use. This can be tested in a comparative study. During the last few years, Handelsman et al. (2004) and others have emphasized the need for "scientific teaching" — the application of scientific research methodology by faculty to their own teaching. Future plans for TIEE include expansion of this scientific research component.


Research Team:

Wendy Bigler
Department of Geography and Environmental Resources
Southern Illinois University Carbondale
Carbondale, IL
Courses: Earth's Biophysical Environments, Environmental Conservation, Field Methods

David L. Boose
Biology Department
Gonzaga University
Spokane, WA
Courses: Human Ecology, Introduction to Ecology

Richard L. Boyce
Department of Biological Sciences
Northern Kentucky University
Highland Heights, KY
Courses: General Ecology

Judy Bramble
Department of Environmental Science
DePaul University
Chicago, IL
Courses: General Ecology

Alan B. Griffith
Department of Biological Sciences
University of Mary Washington
Fredericksburg, VA
Courses: Plant Ecology

Elizabeth Hane
Department of Biological Sciences
Rochester Institute of Technology
Rochester, NY
Courses: General Ecology, Conservation Biology, Plant Ecology

Robert Humston
Department of Biology
Virginia Military Institute
Lexington, VA 24450
Courses: Plant Biology, General Ecology and Evolution

Laura A. Hyatt
Biology Department
Rider University
Lawrenceville, NJ
Courses: Ecology, Life Science: Eco-botanical Emphasis, Principles of Biology

Gail Langellotto
Department of Biological Sciences
Fordham University
Armonk, NY 10504
Courses: Ecology lecture and laboratory

Elena Ortiz-Barney
Life Science Department
Mesa Community College
Mesa, AZ
Courses: Environmental Biology

Alissa A. Packer
Biology Department
Susquehanna University
Selinsgrove, PA
Courses: Ecology, Evolution and Heredity, Issues in Biology

Tim Parshall
Biology Department
Westfield State College
Westfield, MA
Courses: Environmental Biology

Christopher M. Picone
Biology Department
Fitchburg State College
Fitchburg, MA
Courses: Ecology, Environmental Science

Jennifer M. Rhode
Department of Biological and Environmental Sciences
Georgia College and State University
Milledgeville, GA
Courses: Introduction to Environmental Science, Ecology, Community Ecology, Marine Science

Jaclyn Schnurr
Department of Biological and Chemical Sciences
Wells College
Aurora, NY
Courses: Ecology, Intro Bio II: Biology of Organisms