Is the best astronomy education research ‘grey’?

Tim Slater, University of Wyoming, tslater@caperteam.com

It’s a reasonable question to pose, “how do I quickly learn about the best that astronomy education research has to offer?” Exhaustive reviews of astronomy education research (Adams & Slater, 2000; Bailey & Slater, 2003, 2005; Slater, 2008), clearly position astronomy education research, AER, as a its own scholarly discipline existing within a rich field of robust student misconceptions and varied instructional strategies designed to intellectually engage students. Indeed, the publication of National Research Council publication, Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering (NRC, 2012) highlights astronomy as one of the principle disciplines of education research, elevating it to the same level as physics education research, geoscience education research, chemistry education research, engineering education research, and biology education research (listed in no particular order). Yet, if it is so important, where is all the astronomy education research?

Unfortunately, because AER is still a fledgling field in comparison to its older brother of physics education research, PER, much of the AER created scholarly knowledge is just beginning to show up in refereed journals. Beyond the some 150 studies summarized in reviews from refereed journals (reviews cited above), most of astronomy education research seems just incredibly difficult to find. Some efforts like SABER (http://astronomy.uwp.edu/saber/) have worked to gather disparate astronomy education research articles in journals beyond the Journal of Astronomy & Earth Sciences Education (http://JAESE.org) or the deceased Astronomy Education Review (http://aer.aas.org), but it just seems like more should be out there, doesn’t it?

Most of astronomy education research that currently exists is hiding in deep in what is known colloquially as the “grey literature.” Grey literature is the scholarly work that has been done and presented at conferences or exhaustively written up in dissertations, but never formally published in refereed journals. For example, at every American Astronomical Society and American Association of Physics Teachers meetings, there are special sessions, both oral and poster, overflowing with astronomy education research scholarship reports. Bailey (2010) reports that in a recent search of the SAO/NASA Astrophysics Data System (ADS), a query of “education” in abstracts for oral or poster presentations at American Astronomical Society (AAS) meetings yielded more than 1300 abstracts since 1992; nearly 600 of those were from the year 2006 or later. And there are similarly impressive numbers of astronomy education abstracts, both invited and contributed from meetings of the American Association of Physics Teachers sub-section now called space science and astronomy. This is the domain of the grey literature.

The best example of the AER grey literature is probably the most well-known study of understanding in astronomy ever done. It was publicized by Philip Sadler and is presented in the video A Private Universe (Schneps 1989). The video begins with clips of interviews with several alumni, faculty, and graduating seniors from Harvard University. Of the 23 individuals interviewed, 21 could not give a scientifically acceptable explanation for the cause of the seasons or the phases of the Moon. Although the misconceptions so cleverly illustrated by A Private Universe were garnered through methods that probably do not conform to the rules of reliability and validity that define today’s DBER, the video’s widespread influence cannot be understated. The Private Universe video served as wake-up call for the astronomy education community, much like Hestenes and colleagues (1992) Force Concept Inventory served as a rallying point for PER. For many, Private Universe marks a major milestone in the evolution of AER. Yet, nowhere does Private Universe show up as a refereed journal article.

As another example, consider the Test Of Astronomy STandards, the TOAST. Conceived of by Stephanie Slater and colleagues, this pretest-posttest, multiple-choice survey of astronomy knowledge is widely used across the international astronomy education research community. Yet, its only formal citation prior to a recent formal publication (now available online for free at S. Slater, 2014) is from an appendix of the book, “Discipline-Based Science Education Research: A Scientist’s Guide” published by Pono Publishing. Books are only sometimes considered by university tenure and promotion review committees as refereed scholarship, yet often hold a treasure trove of wonderful work. It is worth remembering that sometimes AER scholars have so many great projects going on that they just don’t get around to formally publishing everything that is meritous. In fact, this isn’t true of just AER, but is a rarely whispered truth to all of scientific scholarship that we rarely tell students about.

If you really want to see the magic of the grey literature, consider dissertations. Dissertations are a secret place where exhaustive reviews are found. It is often in the second chapter of astronomy education research dissertations where work is widely reviewed in the dreaded literature review chapter. Sometimes found as PDF’s at ProQuest.com, many astronomy education research dissertations review tens, if not hundreds of studies, conference presentations, poster contributions, and other dissertations. Together, this represents a tremendous amount of work reviewing, synthesizing, and recasting previous work that is well worth the time to hunt down, especially if you are trying to get into AER.
So, if you are trying to find the very best and most influential astronomy education research work and aren’t having much luck, you need to know that you are not alone. Most Ph.D. dissertations are never published, and an even smaller percentage of conference presentations end up as refereed journal articles. Just because something never makes it into a formal refereed journal, doesn’t mean that it is bad work; in fact, there are plenty of published refereed journal articles that represent bad scholarship. Rather, it just means that the scholarship cycle was never completed. In reviewing astronomy research, you need to be a critical consumer of the research you are considering and realize that the formal citation rate of the journal you are looking at only is one aspect you should use to judge its relative importance.

---

REFERENCES CITED:

Adams, J. P., & Slater, T. F. (2000). Astronomy in the National Science Education Standards. Journal of Geoscience Education, 48(1), 39-45.

Bailey, J.M. (2010). Astronomy Education Research: Developmental History of the Field and Summary of the Literature. Paper commissioned by the National Research Council, http://www7.national-academies.org/bose/DBER_Janelle_Bailey.pdf

Bailey, J. M., & Slater, T. F. (2003). A review of astronomy education research. Astronomy Education Review, 2(2), 20-45. doi: 10.3847/AER2003015.

Bailey, J. M., & Slater, T. F. (2005). Resource letter AER-1: Astronomy education research. American Journal of Physics, 73(8), 677-685

Hestenes, D., Wells, M. and Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30: 141-158.

National Research Council (2012). Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering, 2012, http://www.nap.edu/catalog.php?record_id=13362

Test Of Astronomy STandards, TOAST. In “Discipline-Based Science Education Research: A Scientist’s Guide, 2^nd ed” by Slater, Slater, Heyer, & Bailey, published by Pono Publishing, http://tinyurl.com/pcmkly4

Slater, S.J. (2014).  Development and validation of the Test Of Astronomy STandards– TOAST. Journal of Astronomy & Earth Sciences Education, 1(1), 1-22.

Slater, T. F. (2008). The first big wave of astronomy education research dissertations and some directions for future research efforts. Astronomy Education Review, 7(1), 1-12. doi: 10.3847/AER2008001,

Schneps, M. P. 1989, A Private Universe [Video], Found at http://www.learner.org/resources/series28.html

WikiPedia entry for Grey Literature, http://en.wikipedia.org/wiki/Gray_literature