Engaging with astronomy has traditionally been an intensely visual experience. Vision allows us to notice the Moon, planets, and stars in the sky. We use vision to peer through small telescopes at the rings of Saturn and (aided by robots) explore the Martian landscape. Individuals with Visual Impairments (VI), including blindness and low vision, are largely left to experience astronomy vicariously through the shared perceptions of others. However, recent developments in audio- and tactile-based programs (e.g., Ferguson 2016; Usuda-Sato et al. 2019) offer new opportunities in astronomy education for students with VI. The planetary science field of astronomy can now offer students, teachers, and researchers the ability to explore by touching representations of the surfaces of many different objects in our solar system.

In this article we describe elements of a 12-month curriculum we developed that uses 3D tactile models and engages students in meaningful science and engineering practices as they explore different aspects of planetary science.

Innovations in Tactile Experiences

Two important developments of the last decade make it possible for us to offer high-quality experiences that let all students interact tactilely with planetary surfaces: (1) the availability of high-resolution topographical data from recent spacecraft missions to various objects in our solar system, including planets, moons, asteroids, and comets and (2) the rapid advance in 3D-printing technology and its dramatic decrease in cost.

We have developed low-cost techniques for designing and replicating 3D tactile models related to planetary exploration. In our workshop at the Lunar and Planetary Laboratory of the University of Arizona, we process recent spacecraft data into digital 3D-printable prototypes. We then use a molding and casting technique to rapidly reproduce models from the prototypes. Our work includes three distinct types of structures that are shown together in Figure 1. We produce monolithic models of planetary terrain, such as volcanos, canyons, and impact craters (top three rows of Figure 1); hollow hemispherical models of objects such as the Moon and Mars, with interior tactile elements (second row of Figure 1); and spacecraft kits that we customize for tactile rather than visual assembly (first row of Figure 1). A detailed description of our process for prototyping, molding, and casting these and other tactile models—along with the full digital archive of our 3D-printable models—can be found in a subsequent paper (Kortenkamp et al. 2022).

POEM: Project-Based Learning Opportunities and Exploration of Mentorship

The focus of POEM is to better understand and further advance the awareness and resilience of STEM-related careers for middle and high school students with VI. The project aims to bridge STEM skills acquired in school with out-of-school experiences to build each student’s capacity for recognizing and pursuing STEM-related higher education and careers through the use of Project-Based Learning (PBL) and enriching mentorship experiences.

Within POEM, we implement three areas of STEM engagement for our VI students: (1) a 12-month PBL experience based on Next Generation Science Standards (NGSS; NGSS Lead States 2013); (2) pairings with university student mentors who are STEM majors; and (3) connections to adult mentors with VI who work in STEM-related careers.

The 12-month PBL experience of POEM begins with a weeklong in-person Readiness Academy during which participants stay at the University of Arizona’s Sky Center facility on the summit of Mount Lemmon in Tucson, Arizona (see Figure 2). Following this experience, the participants return home and begin the series of monthly remote-learning activities during which they receive packages in the mail with 3D tactile models and a STEM engagement curriculum. The finale of POEM is a second week of in-person experiences that involve living as a college student on the campus of the University of Arizona and engaging in an Enrichment Institute (see Figure 3).

In this article we focus exclusively on the monthly remote-learning PBL curriculum but here very briefly touch on various levels of collaboration that are embedded into our overall project. POEM student recruitment, specialized/individualized adaptations of some materials (e.g., production of braille documents and generation of 2D tactile diagrams, see Park et al. 2021), mentor training, and educational data collection were conducted by faculty and graduate students in our College of Education’s program for certification of Teachers of the Visually Impaired (TVI). Our PBL science curriculum development, alignment with NGSS, design and production of 3D tactile materials, and creation of hands-on STEM activities were conducted by faculty, graduate, and undergraduate students in the College of Science, primarily in the Lunar and Planetary Laboratory. Close interaction was maintained between these and other groups.

POEM Student Recruitment and Demographics

Recruitment of POEM students took place through a comprehensive posting on the project’s website, mailings to TVIs in Arizona and surrounding states, and presentations at multiple conferences in the VI field. POEM students were required to meet the following eligibility criteria before selection to participate; (1) be in grades 7–11, (2) have an individualized education program, (3) be independent in their self-care, (4) have academic skills within one year of grade level for reading and writing, (5) be within two grade levels for mathematics, and (6) have interests in learning STEM or pursuing STEM-related careers. For our 2020 cohort we recruited 14 students. Their demographics and previous mentorship experience are shown in Table 1.

The Curriculum – Combining PBL and NGSS

Meeting diverse student needs in STEM learning can often be addressed by adopting a PBL strategy (Guven and Duman 2007; Han et al. 2015; Holthuis et al. 2018). PBL is an effort to engage students in an inquiry-driven STEM process, with extended activities centered on a common theme. In PBL the interests and motivations of learners become the primary focus (Tal et al. 2006). Traditional means of teaching, such as lesson plans, predefined problem sets, and exams are reduced and/or replaced in PBL by increased discussion, mentoring, advising, and creating final products or artifacts such as presentations.

The Learning Outcomes of POEM are such that students who engage with and successfully complete our program will develop:

1. Self-confidence and independence in their ability to use technology as a resource to obtain, evaluate, analyze, and interpret data and information.
2. Experience in multiple methods of communication, including writing/reading, speaking, interactive conversation, as well as formal presentations using props, tools, demonstrations, and activities for an audience.
3. Critical-thinking skills, techniques to explore tactile materials and formulate questions, and the methods they need to effectively communicate their work to a wider non-science audience, including people without VI.
4. Understanding how astronomers use comparative analysis to study core ideas in the Earth and space sciences, particularly related to planets, moons, and small bodies in our solar system.

Each of our monthly curriculum segments is designed to address selected areas of the NGSS, with an emphasis on science and engineering practices. Although the specific science content area of POEM is in the planetary science field of astronomy, our PBL structure is not necessarily designed nor intended to focus on disciplinary Earth and space science core ideas within NGSS. Rather, the objectives of our curriculum are for students to gain experience and understanding in (1) the nature of science and (2) the practices employed by scientists and engineers. Although this happens within the context of space science, the basic structure of POEM could be used in other STEM areas.

The POEM curriculum was developed around crosscutting concepts in the NGSS that include (1) scale, proportion, and quantity, (2) systems and system models, (3) stability and change (understanding processes that operate over long expanses of geologic time), (4) structure and function, and (5) energy and matter. In addition, the interdependence of science, engineering, and technology is addressed within our curriculum. Through the PBL activities and interaction with their mentors, POEM students are expected to demonstrate proficiency in several overlapping areas of the eight science and engineering practices within NGSS (NGSS Lead States 2013), including:

1) asking questions and defining problems,

2) developing and using models,

4) analyzing and interpreting data,

5) using mathematical and computational thinking,

6) constructing explanations and designing solutions,

8) gathering, evaluating, and communicating information.

In addition, there are implicit connections to the Nature of Science elements of NGSS, including how science laws, models, mechanisms, and theories explain natural phenomena.

PBL Designed Around a Unifying Theme in Planetary Science

Craters formed by the impacts of asteroids and comets are the overwhelmingly dominant geologic characteristic of the surfaces of the inner planets and the Moon, with the sole exception of the surface of Earth. When planetary scientists explore the fundamental differences between these objects, we often engage in comparing and contrasting the numbers and structures of impact craters on their surfaces. With this in mind, we designed the entire PBL curriculum around a unifying thematic question in space science, “What can we learn about Mars by studying impact craters from asteroids and comets?” To address this question, the monthly segments use a progression of 3D tactile models that begin with exploring a pristine impact crater in North America, then impact craters on the Moon, and finally impact craters on Mars. Interspersed within the models of planetary terrain are tactile kits of the real spacecraft used to explore the features. The associated activities and student presentations are designed to build upon each other over the course of the yearlong curriculum, with ongoing support from the university mentors.

Structure of the monthly segments within the PBL architecture

Resources distributed to students and mentors each month (electronically and hardcopy, including braille) include:

1. Tactile models (including some 2D tactile graphics when needed)
2. Brief introduction to STEM concepts related to the tactile models
3. Progression of activities and prompts for participant presentations

Expectations of students each month:

1. Curiosity Based Inquiry - recording and sharing a “List of Wonders” related to each monthly segment
2. Gathering and Recording Data - logging of measurements, findings from the activities, and results of directed online research
3. STEM Communication – a brief 1- to 10-minute recorded presentation (e.g., video, podcast, etc.) to a friend, family member, and mentor and submitted to POEM staff

Monthly Segment Progression

As participants progress through the monthly segments, there is increasing complexity to both the activities and the expectations, requiring students to recognize and formulate questions based on their curiosity and learning in previous segments. In addition, the progression includes an increasingly complex presentation to their friends, family, and mentors each month. For example, the presentations initially are created through audio recordings only. They advance to video presentations that include developing a demonstration using a tool, developing a simple activity for the audience to perform, and finally culminating with a presentation using both a demonstration and a complex audience activity. Support and feedback from university mentors each month help students refine their communication skills. The goal is to prepare each student for engagement in the POEM Enrichment Institute when they return to Tucson at the end of the year. During this weeklong experience, participants collaborate in pairs to prepare and give 15- to 20-minute presentations about the science projects they work on during the week.

Table 2 lists the monthly POEM segments with the tactile models included and the NGSS focus areas. All segments emphasize the NGSS practices of asking questions, collecting and analyzing data, and communicating information, in addition to the listed focus area. (Note: Segments 9–11 occurred as the COVID-19 pandemic was emerging. These segments were altered to reduce student anxiety as schools moved online. Segment 11 in particular was conducted as a live-online whole-group discussion and “unboxing” of the Curiosity rover kit.) In the Supplemental Resources section we provide abbreviated examples of two monthly activities (Activities #1 and #5) to better illustrate how the NGSS practices are addressed (the complete unabridged curriculum can be found in Supplemental Resources, along with the timeline used for implementation).

To help represent the structure of our student presentations, Figures 4 and 5 provide still-frames from two different presentations. The example in Figure 4 (from Segment #5 – Two Faced Moon) involved responding to a prompt asking students to include a physical demonstration to present a technique they developed for determining the thickness of the Moon’s crust in their hemispherical models. Presentations reached a peak in complexity in Segment #8 – Martian Craters Tell Tall Tales. Here, students were prompted to develop a complex activity an audience could perform with them while watching their presentation. Figure 5 shows still-frames from a representative submission for this segment. Using this type of escalating PBL approach taps into the growing confidence and energy of students as they progress to higher levels of understanding and engagement.

Evaluation of POEM

POEM is funded by the National Science Foundation’s K–12 ITEST program and undergoes ongoing internal and external evaluation of the various components. This section details the data collection and findings for the following internal evaluation questions:

1. How did the programmatic changes implemented between the 2019 pilot cohort and the 2020 cohort affect retention of POEM students?
2. How did the monthly PBL activities influence student achievement of our Learning Outcomes (described earlier) in terms of each student’s level of …
   1. formal STEM communication and organization?
   2. energy and confidence in STEM communication?

Student Participant Retention – Learning From Our Pilot Year

During our 2019 pilot year, a cohort of 11 students was recruited along with 11 university mentors and 11 industry mentors to give each student one-on-one interaction with mentors. Pilot year students were asked to communicate and individually submit monthly assignments through the university course management system and asked to interact with their mentors independently without significant facilitation from POEM staff, and interact as a community through a Facebook group. Only 5 members of the initial 11 students in the pilot cohort remained engaged for the entire year, a retention rate of less than 50%.

In an effort to improve engagement and retention, we modified several of POEM’s monthly communication aspects for the 2020 cohort. We grouped the 14 students into seven pairs that were each assigned a university mentor and an industry mentor based on the students’ indicated areas of STEM interest. WhatsApp was also established as the primary communication medium to facilitate more convenient formal and informal interaction throughout the year including scheduling meetings to discuss and work on activities, asking questions, submitting requested journals and links to presentations, and fostering a more cohesive community. Each WhatsApp group had two students, a university mentor, and an industry mentor. An umbrella WhatsApp group was created for all students, mentors, and POEM staff to bring together all involved under one online community. When needed, we used other online modalities, such as email, text, and YouTube, for communication and presentation submissions.

These changes made it possible for students to be more active and engaged in POEM and fostered a much stronger sense of community among our entire group. Of the 14 students who started in the 2020 cohort, 11 remained engaged at the end of the year. This approximately 80% retention rate significantly improved upon the less than 50% retention in the pilot year.

Student Participants’ Communication, Organization, Energy, and Confidence

During the first eight monthly activities, the student participants submitted a recorded STEM presentation that addressed specific prompts, such as including introductions, conclusions, physical demonstrations, and audience activities. We developed a rubric, shown in Table 3, to evaluate these presentations.

Our rubric was developed after consulting Capraro et al. (2013, pp. 155–157) and includes three different categories related to several of the eight NGSS science and engineering practices described above (see also NGSS Lead States 2013). These include Communication and Organization in STEM (related to NGSS practices 5, 6, 8), Energy and Confidence in communicating ideas and information (NGSS practice 8), and Addressing the Presentation Prompt (NGSS practices 2, 5, 8). Where Capraro et al. (2013) use a broad and highly subjective “speaker conveys confidence” grading criteria, we tried to be more explicit in evaluating confidence and energy by using arguably less subjective elements of tone, rate, and clarity of speaking.

Each of the 10 grading areas in the rubric was given a score value ranging from 0 to 4. A score of 0 only applied to the Addresses Prompt area and indicates the grading area was missing (e.g., the specific prompt for the presentation was not addressed at all, such as no physical demonstration included), a 1 indicates poor performance, up to a 4 for an excellent performance. If the Grading Area was not applicable for a particular presentation, it was left blank (e.g., no gestures or body language in voice-only presentations).

Of the initial 11 participants in the 2020 cohort that remained engaged with the program at the end of the year, 10 were actively submitting the requested STEM presentations throughout the year. One of the 11 remained engaged and continued submitting monthly reflections of mentor interactions and comments regarding science activities, but did not submit the requested STEM presentations. The 10 active participants submitted 64 STEM presentations in response to the monthly segments, for an 84% completion rate out of the total 80 possible presentations. Three participants submitted all eight presentations and two others submitted seven out of eight.

Each presentation was evaluated by four members of our POEM team (one planetary scientist and three in the TVI program) using the rubric in Table 3. These four scores were then averaged. Figure 6 shows the results for the five students who submitted at least seven of the eight monthly presentations. While the amount of data represented by our evaluations of these students is likely too small for meaningful statistical studies, there are indications of improvement in the Communication and Energy/Confidence scores of these five students. The left panel of Figure 6 displays a tiered characteristic, with the last five averaged monthly Communication scores about 0.5 points higher than the first three averaged monthly scores. Energy/Confidence scores in the right panel of Figure 6 display characteristics of an improving trend across the eight monthly scores, from an initial low of about 2.5 in August to about 3.25 in April.

Conclusions and Discussion

In our PBL structure we avoid traditional means of STEM assessment, such as exams and/or problem sets. Instead, other artifacts such as presentations serve a critical role in demonstrating achievement of our student learning outcomes. Our rubric allows us to document and analyze student progress in communication, organization, energy, and confidence. The data reported in this article demonstrate promising trends where students who completed most presentations (at least 7 of the 8) improved over the year. Our developed rubric and reported findings will inform the research team as POEM is further adjusted for future cohorts. Another implication of POEM data relates to the recognition of the important role informal communication and a sense of community play in increasing student retention, engagement, and enthusiasm in STEM.

Looking beyond students with VI, we suggest that our 3D tactile resources and PBL/NGSS activities can be adapted by science teachers for conventional in-class instruction.  For example, one member of POEM uses the tactile models and some activities in a university-level general education science class. A high school science teacher connected to POEM has integrated most of our tactile models into their science curriculum for sighted students.

Because of the growing access to 3D printers in schools, the approach presented here has the potential to be implemented more broadly with both VI and non-VI students. While our project used a customized molding and casting process, a simple Google search will reveal abundant online archives of 3D printable files available from NASA and many other organizations (e.g., sites.google.com/view/microbiologyfortheblind and btactile.com). Schools with maker-spaces could easily print 3D models to be used in classrooms for both visual and tactile exploration.

We wish to conclude by emphasizing a rather surprising aspect of POEM that only became apparent after the COVID-19 pandemic forced nearly all schools to go online. Our STEM education experience was conceived and implemented pre-COVID, when remote learning was still very much a novelty for most students in the grade range we work with. This is especially true for students with VI. Yet, the 2020 cohort of POEM student participants was remarkable for both their retention rate and their assignment submission rate. Eleven of 14 students who started the program remained engaged after a full year. Of these 11 students, 10 were actively submitting monthly presentations and ultimately submitted 64 of 80 presentation assignments, for a completion rate of 84%. Considering that POEM was a yearlong, extracurricular, remote-learning science program, this level of consistent engagement is astonishing to us. We believe this speaks to the desire and need for more programs of this kind to engage students with VI in STEM experiences.

Acknowledgment

The authors wish to thank members of the POEM external advisory board. This study was performed at the University of Arizona and funded by the National Science Foundation’s ITEST program (award #1657201). Contents of this publication do not represent the official views of NSF. POEM is approved for human subjects research at the University of Arizona by our Institutional Review Board (IRB #1702248263). In compliance with this approval process, we obtained assent forms from students as well as consent forms from their parents and mentors.

Stephen Kortenkamp is an associate professor of practice at the University of Arizona Department of Planetary Sciences – Lunar and Planetary Laboratory in Tucson, Arizona. Jinseok Park and Tasnim Alshuli are doctoral students, Garrison Tsinajinie is an assistant professor of practice, Sanlyn Buxner is an associate research professor, Irene Topor is an associate professor of practice (emeritus), and Sunggye Hong is an associate professor, all at the University of Arizona College of Education in Tucson, Arizona.