Using Evaluation to Improve Educational Experiences
Regardless of the setting in which learning takes place, identifying the goals and intended outcomes of an educational experience, then measuring how well that experience achieved them, is becoming more commonplace and is often essential. However, many traditional forms of evaluation and research—such as action research, participatory evaluation, professional inquiry, and reflective practice—require longer time lines, more intense planning, or hiring external consultants, all of which are often impractical in many educational settings.
Recognizing the need for a different approach to evaluation, the National Informal Science Education Network (NISE Net) developed Team-Based Inquiry (TBI) in 2011 and refined the process over the next four years, ultimately resulting in the publication Team-Based Inquiry: A Practical Guide for Using Evaluation to Improve Informal Education Experiences. Although TBI was originally crafted to help informal science education professionals develop the skills and tools to incorporate evaluation and data-informed decision-making into their work, it is also applicable to other learning settings. In fact, TBI is a particularly useful process for teams of educators from multiple organizations to make program evaluation a truly collaborative process.
“Team-Based Inquiry empowers education professionals to get the data they need, when they need it, to improve their products and practices and create successful educational experiences” (Pattison, Cohn, and Kollman 2014, p. 5).
The key characteristics of TBI are that it is:
TBI uses an ongoing, cyclical professional inquiry process defined by straightforward, actionable steps to help any individual or team of educators gather and reflect upon data that will help them move their project forward.
The TBI process seeks to identify and prioritize questions, investigate those questions by collecting relevant information, reflect on how and whether the information answers the questions, and take action to improve the learning experience being studied. This process is particularly useful for short programs, such as hour- or week-long experiences, STEM (science, technology, engineering, and math) experiences that have the ability to be modified, and programs facilitated by educators, volunteers, or others who can modify the facilitation practices to strengthen the experience.
The four stages of the TBI process are further described below:
It is important to document your team’s process while using TBI. TBI is often iterative, because the process of doing it often leads to new inquiry questions that your team will want to investigate. Further, your findings may be shareable. Consider capturing your questions, data collection process, reflections, and actions in a short report to share with others within your organization and perhaps even external stakeholders. Adequate documentation will also help prevent the need to duplicate your efforts in the future.
The TBI process is intended to be flexible and adaptable—check out the resources and use what works for you!
Sarah Cohn (SCOHN@auroraconsult.com) is principal consultant at Aurora Consulting in Minneapolis, Minnesota. Previously, she was at the Science Museum of Minnesota in St. Paul, Minnesota.
Cousins, B., S. Goh, S. Clark, and L. Lee. 2004. Integrating evaluative inquiry into the organizational culture: A review and synthesis of the knowledge base. The Canadian Journal of Program Evaluation 19 (2): 99–141.
King, J. 1998. Making sense of participatory evaluation practice. New Directions for Evaluation 80: 57–67.
National Research Council (NRC). 2009. Learning science in informal environments: People, places, and pursuits. Washington, DC: National Academies Press.
Pattison, S., S. Cohn, and L. Kollmann. 2014.Team-based inquiry: A practical guide for using evaluation to improve informal education experiences. 2nd ed. Boston: National Informal STEM Education Network.
Preskill, H., and R. Torres. 1999. Evaluative inquiry for learning in organizations. Thousand Oaks, CA: Sage Publications.
Journal ArticleIs a Framework of Support Enough?
While undergraduate research is known as a high-impact practice, little research has been conducted for the online educational setting. Early research...
Journal ArticleAn Examination of Constructivism, Active Learning, and Reflexive Journaling and Their Independent and Combined Effects on Student Acceptance of Biological Evolution
Instruction that increases acceptance of evolution is essential to effective biology instruction, but instruction about evolution is not consistently ...
Journal ArticleSTEM Faculty Institute
As change agents at our university, we sought to facilitate a transition within our STEM college toward more extensive use of evidence-based instructi...