Tim Slater, University of Wyoming, Tim@CAPERteam.com
This is the sixth in a series of about six very long blog posts on innovative college astronomy teaching written with the straight forward goal –> to provide busy faculty with easy-to-implement teaching strategies that dramatically improve the student learning experience. The first, second, third, fourth and fifth blog posts are available online.
My recommended strategies at helping you do a more sophisticated job of teaching astronomy and planetary science to college and university students up until this point has been to equip you with the basic tools and recipes you need to compellingly convince your students you are a first-class teacher who has the ultimate goal of helping all students learn as a result of intellectually investing in your class. This thinking is based on the underlying notion that if students perceive that they are measurably different after your astronomy class then you, dear colleague, will be rewarded with overwhelmingly positive and enthusiastic end of class student evaluations. There is, however, one key linchpin to this entire system that you cannot afford to mess up if you want to receive really outstanding course evaluations—students must be fully aware of the specific things you are doing to support and coach them into earning high grades. The tool here is a winning syllabus.
To have the best chance at student learning success and high end-of-course evaluations, your students have to both know and be able to describe why your class helps them learn in order to report back how wonderful your class is on their end-of-course evaluations of you. Helping students know what you are doing for them requires an abundantly clear and easy to understand syllabus. After all the hard work you’ve put into your class, be warned, it is far too easy to overlook the nearly immeasurable value of a carefully constructed and frequently revisited syllabus.
The Syllabus as a Structurally Organized Pathway
Your syllabus serves at least three critical functions that are too often taken for granted by novice professors. The first function is that a syllabus manages student expectations. Any situation in which students’ expectations are in conflict with reality, there is a potential for you to receive negative course evaluations.
Allow me to explain. By and large, most students’ experience with the vast majority of their college courses are characterized by laboriously hand-writing out detailed notes during several lectures each week and then memorizing those notes to reproduce on several exams. Certainly, there are a highly limited number of memorable exceptions to this, but such a description applies to the vast majority of the 40-some courses students take during their undergraduate career. Because this is what most students experience in college, this is what they are expecting from your class.
Moreover, students know that far too many faculty start each semester with a bold proclamation to their students that their course is uniquely different from all the others college courses and students will have to apply high-level, critical thinking skills to earn a high course grade. However, because students hear such empty statements semester after semester, students have come to learn that such proclamations is most often largely false theatrical bravado, and that success in most college courses can be achieved frequently by simply memorizing lecture notes and done so without even purchasing the required textbook. In other words, students have been trained that most courses are alike despite what their professors say on the first day of class.
First: Manage Student Expectations
If your course really is going to be different than all the other courses students are taking, if you are really going to convince your students that you have created an organized pathway for them to succeed, and if you are really going to serve as a coach in guiding students to learn and love science, then you dear professor must prove your intentions to students repeatedly throughout the term. The best way to demonstrate that your course really is different—and more worthwhile—than their other classes is to follow a syllabus that holds you true to your word and constantly engineer students’ behavior to be success-oriented and to use and review the syllabus during the semester.
You would be well advised include a clear statement of why you designed the class the way you have and revisit this rationale frequently. Consider as an example,
“Astronomy is a verb. I do not believe you can learn and love science simply by watching the professor talk about it, any more than one can learn to swim only by watching a swimmer talk about swimming. In contrast, I believe that you can best learn astronomy—and love science—if you DO astronomy. Therefore, I have designed this class around you completing a series of experiences where you look at what astronomers look at, measure what astronomers measure, and use scientific models the way astronomers use them. Although there will be lectures for you to attend to throughout the class, the majority of your intellectual effort needs to be spent not on memorizing notes, but on reviewing the activities we do during class.”
If you like this statement, you should feel free to use it or adapt in any way you see fit in your syllabus.
The goal here is to help students know where they should be placing their cognitive energy. At the same time, your end-of-course evaluations will be best served if you talk about your approach about once per week to remind students why you are doing what you are doing. Students are often taking many other courses and can get confused. The icing on the cake here is for you to not only talk about your class, but to frequently show students sample exam questions several times each week and clearly indicate how the questions are best answered, not by memorization, but instead by using the same reasoning strategies that they use in their learning activities and discussions.
Students are expecting to spend their intellectual energy on memorizing and if that will not help them earn high marks, you need to tell and consistently illustrate to students what you value. Furthermore, you need to “put your money where your mouth is” and be sure that your exams clearly demonstrate that you value understanding more than memorization. Your end-of-term evaluations will benefit if you repeatedly shown sample exam questions which clearly illustrate that putting cognitive energy toward understanding and engaging in learning activities is more important than memorization notes in determining their grades.
As it turns out, students might enter your class on the first day with some profound misconceptions about what your class is actually about. Introductory Psychology students might think your class is about helping them be more emotionally confident. Introductory Geography students might think your class is designed to help them memorize the names and locations of countries around the world. Astronomy students often mistakenly believe that astronomy classes are about memorizing constellations and identifying stars in the night sky. If the astronomy class is one mathematical equation after the next and little class time is allocated to learning constellations, your end-of-class evaluations will suffer because students’ expectations were not managed by you—even if you have the best class in the world. No matter what class you teach, you need to be very clear about what your class is about, and what it is not about.
Second: Provide a Clear Road Map
The second reason you need to provide and follow a clear syllabus is that your syllabus can provide a day-to-day recipe and clear road map to follow for students’ success. A fruitful syllabus clearly identifies for students what they are supposed to be doing and learning each and every class meeting day (not each week broadly or each vague unit). At the same time, the best syllabi help students know what behaviors in which they are supposed to engage. Students are often taking several classes and it is all too easy for them to lose track of what they are supposed to be doing. Some faculty find that giving students a weekly checklist to follow is really useful in guiding student behaviors toward success.
| Monday Morning Student Success Checklist
Before Tuesday morning, did you:
⎕ Read all of the figure captions in assigned readings so that you understand what experts are paying attention to?
⎕ Review and rewrite your lecture notes more neatly so you can better remember concepts for the exam?
⎕ Understand the correct answers to clicker-voting questions so you know if you need further study?
⎕ Carefully read & highlight last week’s readings so you fully understand the ideas?
⎕ Write answers to assigned end-of-chapter questions to check that your level of understanding is correct?
⎕ Be sure you understand what you were supposed to learn during the last group activity so that you know you are using class time most effectively?
⎕ Create three hypothetical test questions you might be asked on an exam and share with a classmate to check your level of understanding?
⎕ Quickly review readings & notes from two-weeks ago so that you do not forget what you are supposed to be learning?
⎕ Skim & highlight next week’s upcoming assigned readings to best prepare you to learn during class time? |
What an astronomy professor who is trying to get positive end-of-course evaluations needs to avoid is students blaming you for being a lousy teacher as the cause of their poor course grades. A checklist of behaviors allows a professor to honestly say, “if you do all of these things, I guarantee you’ll get high grades.” This isn’t an empty guarantee: If students follow a consistent plan, they will earn high marks. And, if students do not follow the plan, then students who fail have a sensible reason about themselves and their behavior they can use to justify their failure, besides reporting that they had a lousy teacher.
Novice learners—your students—depend on you to have sufficient knowledge of the course content and its structure to build a clear and easy-to-follow pathway to successfully learning the course material. Students have little idea what actually learning material in your specific discipline looks like. Should your students be highlighting their course textbook and making flash cards for memorization? Should your students be working problems in the back of the book and even making up problems of their own to swap with classmates? Should your students be seeking out and listening to lecture from other scholars in your domain? Should students read the assigned readings twice, and critically challenge the conclusions the author is drawing? It certainly isn’t all of these things, because different disciplines require different learning activities from their students. Physics professors clearly do not want their students to compose philosophical essays on why their textbook author is incorrect whereas literature professors sometimes do! Whatever people need to do in your discipline to engage and learn it, your students do not know this, and need you to help them engage in those specific activities.
At the same time, students have no idea how fast to be moving through the material. And, so it seems, some professors seem to have given surprisingly little thought to how fast they should be pacing through the material. My concrete evidence for this is that far too many busy professors tell me at the end of the term that they are madly rushing their lectures to get through all of the concepts that they had planned or are required to cover.
Given all of that preamble, the key idea here is that professors who are highly successful and peacefully prepared teachers purposefully build for students—and themselves—a detailed, daily schedule of course learning activities, distribute to students, and refer to it frequently both by themselves and with their students during class. And, they and their students are committed to sticking to the schedule no matter what happens during scheduled class time.
Imagine a day-to-day scheduled syllabus that includes a five-column table where each row is a class-meeting day. The columns going from right to left are labeled:
| Class Date | Before Class | During Class | After Class | Notes |
A syllabus schedule that really works is sufficiently detailed that not only can an unexpected substitute professor pick up and use in your absence but provides a clear learning pathway students could follow in the absence of an instructional leader.
The most successful professors always make a detailed learning plan for students—and themselves—to follow as a road map, and religiously stick to it even when scheduled class time doesn’t go as planned or, for some unanticipated reason like inclement weather or worse, class gets canceled. This keeps your courses and their teaching under control and out of the crazy-making whirlwind that inevitably happens at universities, causing frantic professors to crumble and endlessly complain. Sound like a lot of work? It is…but there are additional benefits to a highly structured, student-centered syllabus.
Perhaps more important than student behaviors, a highly-structured syllabus makes the professor’s life easier. Professors often feel overwhelmed and incredibly busy, especially near the end of a term. Time management and productivity experts universally agree, that having a highly structured life is not constricting, but is actually liberating. If you explicitly what it is that you are supposed to be doing on a particular day, then when that task is done, you are free to do whatever you want or need to do. Experienced professors know that the best way to get through all the material you want to cover is to have a detailed plan for precisely how you will get there—and to follow it! If you have an organized plan for your students, and an organized plan for you, then everyone is happier.
Third: A Syllabus as a Contractual Written Agreement
The third function is to serve as an apparent legal contract between you and the students. No one wants to have legal battles or extended and awkward grievance committee reviews, but sometimes there are conflicting understandings about what is expected of students and what students expect of their professors. There is no getting around this and college administrators will sometimes say that thinking of a syllabus as a legal contract is the most important reason for professors to have a syllabus. We do not wish to belabor this point, but the frank truth is that college administrators and their accreditors want you to have a strong, legalistic-style syllabus for their administrative benefit, not for your protective benefits or for your students’ learning benefit.
As a result, you should definitely have a clear syllabus to avoid being hassled by these overseers. But, based on what we have argued above, your syllabus is so much more valuable as a philosophical communication tool and a structural management tool. If you manage student expectations about what they will learn and how they will learn it, and if you provide a clear and organized pathway for success, then more of your students will learn to love science than would otherwise happen.
Your Attendance Policy
Are you satisfied with the number of students who attend your class regularly? Is your Friday class meeting disappointingly lower in attendance compared to your Wednesday class session? Maybe it is time to consider implementing an attendance-required policy in your class and give students points (or subtract points) for attendance? Or maybe such a punitive college class attendance policy is not such a good idea. This notion has been debated and re-debated many times over, but I advance that there is actually a better approach.
For my nickels, it stands to reason that, by and large, students who attend class get better grades than those students who do not attend class. Beyond reason, and with a little concerted effort, most science professors can systematically collect sufficient data to show that in their own classes where there is no draconian-level punitive attendance policy, that their students who attend class generally get better grades than their students who do not. Given such logic and evidence, one might quickly assume that better overall course grades will result if more students are attending your class.
As a seemingly unrelated but parallel example, consider that at least one national group of pediatricians became cognizant that school children who have numerous books in their homes excel in school as compared to school children who have few books in their homes. As a result, pediatricians throughout the country have devoted considerable time and money to giving books to low socioeconomic class children who come in for medical treatments and preventative care, with the enthusiastic hope that a larger portion school children would begin to excel in school—all because more books in the home seemed to equal better academic performance. This is a classic case of confusion between correlation and causation, even though the program has been repeatedly lauded as a model program on the floor of the US Congress.
In both perspectives, well-meaning professionals are confusing correlation and causation. Here, what seemed to be the cause—consistent class attendance and books in the home—are symptoms rather than explanatory, causal mechanisms. As it turns out, families in the US that focus their fiscal and human resources bringing books into the home are more often also supportive of high academic achievement. And, similarly, college students who are self- and intrinsically-motivated to attend college classes regularly are also apt to take their overall college learning experiences with more enthusiasm and end up earning better grades in college overall. Moreover, research of college teaching consistently demonstrates that seemingly harsh attendance policies erode students’ motivation, regardless of how good a class is on its own two feet.
So what is a passionately committed college astronomy professor to do? I do not recommend instituting a penalty (or incentive) grade-based attendance policy. For one, such policies have not been shown to work. For another, the record keeping related to enforcement in either case is onerous. But, most important to me, attendance policies can degrade students’ sometimes fragile levels of motivation, which I am trying so hard to nurture so that they can learn astronomy.
Instead, I recommend professors consider first and foremost that each and every day in class, they engineer specific learning activities—clearly aligned with helping students attain high levels of performance in their course grades—that make coming to class highly valued and consistently worthwhile. Group activities, mini-debates, case studies, test-item-creation tasks, among many others, engender active learning experiences that cannot easily be replicated by simply getting the lecture notes from another student. In other words, instead of spending one’s effort on the record keeping related to student attendance, reallocate one’s effort to making class time so valuable a student doesn’t wish to miss a single class meeting!
For myself, I certainly do have a stated class attendance policy, but it is not directly tied to students’ grades numerically—I believe that my students should earn a grade based on what they understand and what they can do. Experience suggests that such non-punitive attendance policy approaches only work if professors purposefully take the time to repeatedly explain why you value class attendance and why students should value it too—certainly on the syllabus and perhaps reminding students every couple of weeks too.
It is worthwhile to tell students during class why their attendance today is valued and what benefit they are receiving today by doing today’s activity. As harsh as it sounds, indeed if students really can pass your class without attending class regularly, then perhaps your class time really isn’t that important and it is time to rethink, re-conceptualize, and reinvigorate what happens every day in your class.
The Syllabus Scavenger Hunt
Does it seem like your students never bother reading the syllabus? A quick Google-image search of “it’s in the syllabus” will reveal countless cartoons-some humorous and some not so humorous-suggesting that students insufficiently paying attention to the syllabus is common place across academia. I really DO want my students to read, understand, and internalize my syllabus, so what is going on here and, most importantly, what can I do about it?
Many students just do not seem to understand the syllabus. The question here is, are students really reading your syllabus? On one hand, it is possible that your students are carefully reading your syllabus; however, perhaps your carefully designed program of study or thoughtfully constructed course policies are simply lost in the sea of first-day-of-class information students are receiving from your class and their other four classes on the same day. The result being that it seems as if students are not reading your syllabus when they actually are; but in sipping from the information fire hose, your crucial information is understandably lost.
On the other hand, it is possible that your students are not bothering to read your syllabus at all. Perhaps students have learned through repeated college learning experiences that syllabus information on average of all their classes is really is not that important and that professors will make emphatic note during class of important dates and policies as they apply over time. The result being that students really do not need to allocate mental energy to ingesting a class syllabus.
Either way, this poses an important problem. There really are important policies in your syllabus you want students to read and internalize. And perhaps more importantly, I have spent a considerable amount of time designing a detailed pathway for success for my students to follow that they need to be explicitly aware of if I am to hope for high end-of-course evaluation scores.
My solution is that I assign an open-book quiz on the critical content of my syllabus that students are required to take-and counts toward their overall course grade-ten days after my class starts. The primary and most obvious advantage of using a syllabus quiz is that students have a graded-task that engages them in understanding my course policies and procedures.
Moreover, there are nuanced advantages to using a Syllabus Quiz. Among them is that students get to have some experience with the testing format I use. I happen to employ a computer-based testing system for all my tests, and having students take the Syllabus Quiz online gets them acquainted with the online system before they take their first content test in that potentially foreign environment. If I didn’t use the online system, but used Scantron-style bubble sheets instead, then that is the format I would use for the Syllabus Quiz. Same thing if I used essay exams: My Syllabus Quiz would be essay style.
| Syllabus Quiz Examples
⎕ What is the course attendance policy?
⎕ What should students be doing during class time?
⎕ What is the fastest way to get a hold of me?
⎕ Which is most influential assignment or test that influences a course grade?
⎕ If students have a family emergency requiring them to leave campus, what should they do?
⎕ When are homework assignments due and how are assignments submitted?
⎕ How often do I need to post in the online discussion group?
⎕ What will the course textbook be used for and is it required? |
Probably the biggest advantage of using a Syllabus Quiz is that I get to emphasize to my students which parts of my course policies and learning plan I have designed for them is most important. This is far more valuable to me than having a documented record that my students knew my official policies in case of a grievance. I really do want my students to know that I have constructed a pathway to master the concepts in my class, and the Syllabus Quiz is but one tool I have in my arsenal to help them succeed.
Easy to Understand Grading Scheme
Students most often appreciate knowing precisely what is required of them in order to earn particular grades. It is naïve to assume that all of your students hope to earn an excellent rating, A-grade in your class: Some students are perfectly happy doing just enough learning to earn an average rating, C-grade.
Traditionally in the US, college students have awarded grades based on the percentage of questions correct on exams, with students who achieve 90% examination items correct earning an Excellent, A-grade ranking.
| Percentage of Correct Answers |
Learning Achievement Ranking |
Grade Assignment |
| 90-100% |
Excellent |
A |
| 80-89.9% |
Above Average |
B |
| 70-79.9% |
Average |
C |
| 60-69.9% |
Below Average |
D |
| < 60% |
Failing |
F |
You might be surprised to learn that adherence to a 90, 80, 70, 60 grading scale is highly discipline-specific. In some science courses, earning a 60% correct and above might earn an A-grade, and 45% correct might be a B-grade. If you don’t follow this typical 90, 80, 70, 60 scale, you need to clearly state that at the outset and remind students periodically through the course. The impact of not clearly explaining your system to students can be profound in that students who culturally assume that getting 60% correct on your exam is a Falling-grade will often abandon your course, even though from your perspective, they were excelling in your class. The bottom line is to be highly cautious about deviating from the traditional 90, 80, 70, 60 grading scale.
Negotiating an Individual Grading Scheme with Your Students
Many professors dread spending time on the first day of class going over their syllabus. Students usually hate hearing professors read the syllabus to them too! But, what if the first day of class could be different? What if instead you could have a different scenario, one in which students are deeply engaged with your explanation of the syllabus?
What if instead you framed your syllabus as the scholarly and intellectual design of how you intend to coach your students through demonstrating mastery of the course topics? Unfortunately, when dealing with human beings, one size definitely doesn’t fit all. And, as with any collaborative team you are responsible for coaching to top performance, your players will have varying strengths and weaknesses. The best coaches know how to leverage what players do well and compensate for players’ weaknesses in order to build a winning program. In other words, if a one-size-fits-all plan doesn’t work to bring out the best performance of a collaborative team of individuals, why would a one-size-fits-all syllabus work for your class?
Consider instead of using a one-size-fits-all syllabus, to adopt an individual ‘negotiated syllabus,’ that differs slightly for each student. A negotiated syllabus still lays out your overall intellectual plan to help students master your course topics but provides critical flexibility to account for your students’ academic (or pragmatic) strengths and weaknesses.
Compare the two hypothetical syllabi grade calculations below:
| Traditional Grading Scheme |
Negotiated Grading Scheme |
| Attendance – 20%
Homework – 20%
Term Paper #1 – 10%
Term Paper #2 – 10%
Exam #1 – 10%
Exam #2 – 10%
Final Exam – 20%
TOTAL EQUALS = 100% |
Attendance – 5% to 20%
Homework – 5% to 20%
Term Paper #1 – 5% to 15%
Term Paper #2 – 5% to 15%
Exam #1 – 5% to 20%
Exam #2 – 5% to 20%
Final Exam – 10%-30%
TOTAL MUST EQUAL = 100% |
In the case of a negotiated syllabus, students can adjust their grade calculation—within clearly specified tolerances—to play to their own strengths and compensate for their weaknesses.
Imagine an older student who has trouble with consistent childcare: This student might benefit from minimizing his grade’s reliance on attendance. Alternatively, consider a student who believes they are a strong writer but a lousy test taker: This student might benefit from limiting their grade’s weighting on exams and increase her grading scheme’s weight for homework and term papers. Because most of us calculate student grades in a spreadsheet or online course management system, it requires surprisingly little time to create a specific grade-calculation-formula for each student based on their choices.
It is important to emphasize that this negotiated formula is to be agreed upon during the first month of class, and not allowed to be revisited later in the semester. Otherwise, this approach is nothing more than an optimized grading scheme invoked late in the semester. I recommend repeated announcements that the negotiated syllabus cannot be renegotiated after the first month.
The goal here is to enhance students’ buy-in and build their ownership in your course design for them and, in particular, to encourage them to understand the course requirements in the syllabus from the very first day of class. Used in this way, the negotiated syllabus strategy can build a cooperative team mentality emphasizing that you want every student to be successful in your class.
Tests Aligned with Lectures & Tasks
Unquestionably, the biggest failure a professor risks when teaching a class is a disconnect between what is asked for on exams and what is being taught. As discipline-specific experts, professors can easily see the connection between exams and class time, but novice students often cannot. As a result, students need to be clearly shown the connections between what happens in class and what performance is asked for on exams. You would be well advised to take class time before and after each exam to discuss this with students demonstrating actual example test questions.
Writing Better Multiple-Choice Questions
Multiple-choice exams are often the least preferable of exam formats, because it can be really challenging for professors to uncover exactly what students now understand and what they still struggle with. At the same time, for those of us teaching large enrollment science classes, we often feel forced to use easy-to-grade multiple-choice exams just because of the sheer size of large courses. Even a single short-answer essay question that takes only 3 minutes each to grade translates into 20 hours of grading when you have four hundred students. For better or worse, easy-to-grade multiple-choice style exams are consistent aspect for many of us. And, if you tend to do a lot of classroom polling, discussing, and voting, having a library of easy-to-use multiple-choice items to use during class is critical. What, if anything, can a busy professor do to increase the cognitive level of multiple-choice items?
One strategy is to reformat visual graphics to have (a), (b), (c), (d), and (e) answers. Imagine saving a copy of a labeled diagram from your textbook into a PowerPoint slide (or whatever your favorite presentation software is that has some drawing capabilities) and pasting white text boxes over the labels. If you use text boxes with letters and a list of terms, this can become a multiple-choice item.
Consider the following examples for asking about complicated ideas and information, using a multiple-choice format:
For the diagram above, the appropriate numbered labels are:
- (a) Castor, (b) Pollx, (c) Capella
- (a) Castor, (b) Pollx, (c) Capella
- (a) Castor, (b) Pollx, (c) Capella
- (a) Betelgeuse, (b) Bellatrix, (c) Sirius, (d) Procyon
- (a) Betelgeuse, (b) Bellatrix, (c) Sirius, (d) Procyon
- (a) Betelgeuse, (b) Bellatrix, (c) Sirius, (d) Procyon
- (a) Betelgeuse, (b) Bellatrix, (c) Sirius, (d) Procyon
- (a) Betelgeuse, (b) Bellatrix, (c) Sirius, (d) Procyon
Another strategy is to take advantage of the higher cognitive levels required to do a sorting task, such as completing a traditional Venn Diagram. But, instead of asking students to write words into an overlapping circle diagram, one could ask students to classify concepts into category A, category B, or both.
| Select (a) for category A: Weather; (b) for category B: Climate; or (c) if it is part of both. |
| 1. Air Temperature on Tuesday |
(a) (b) (c) |
| 2. Humid Subtropical |
(a) (b) (c) |
| 3. Partly Cloudy |
(a) (b) (c) |
| 4. Arid Desert |
(a) (b) (c) |
| 5. Windy, chance of rain |
(a) (b) (c) |
| 6. Polar Artic |
(a) (b) (c) |
| 7. 72° |
(a) (b) (c) |
| 8. 1013 mb (29 inHg) & falling |
(a) (b) (c) |
| 9. Foggy, with low visibility |
(a) (b) (c) |
| 10. Predominantly westerly winds |
(a) (b) (c) |
| 11. Icy |
(a) (b) (c) |
| 12. Glacier melting causes sea levels to rise |
(a) (b) (c) |
Finally, you need to be aware that one of the biggest challenges when creating multiple-choice tests that work are that there is often just so much reading to be done on the part of the student. This is particularly problematic for students whose native language is not English. Sometimes, we inadvertently create an exam that more accurately is testing a student’s reading ability than their actual scientific understanding. To get around this, one strategy is to use essentially the same question over and over again, but only changing one aspect of the questions—and clearly marking which aspect is changing from question to question.
Helping Students do Well on the Final Exam?
“What is going to be on the final exam?” That’s the most common question I get from students this time of year and, frankly, it makes me want to pull my hair out! You see, I’ve spent all term meticulously outlining the major course concepts, carefully leading my students on a connected conceptual journey, leading them up the cognitive mountain to the grand finale that pulls all the ideas together in a symphony the grand masters of composition would find nothing less than awesome. Yet, somehow, students mysteriously ask me to tell them what’s on the comprehensive final exam.
To be honest, I do have some sympathy for my students. Most of my students are taking three, four, or more other classes this term and they have a considerable amount to keep track of. Here at the end of the term, students are honestly trying to figure out how to prioritize their end of the term tasks and get the highest grades they can possible earn. They know—and I know—professors can’t ask a final exam question on every single thing we covered during the term, and students probably would appreciate some assistance in focusing their final exam preparation. And, students are not taking my class in a vacuum from other professors: Students are enrolled in other classes with impending final exams and many of those professors give end of term study sheets. In the end, it is probably a reasonably thing for students to ask for some guidance. So, what’s a professor to do?
Two important ideas are too often forgotten by professors at the term’s end that we all need to keep in mind:
Idea #1: There are big ideas that professors want students to know five years from now
Idea #2: There are factoids and minutia that professors do NOT necessarily want students to remember five years from now.
The problem that pervades the comprehensive final exam realm is that professor’s unconsciously find that is much, much easier—and thus happens more often than not—for professors to write memory-level test questions than it is for professors to create integrated, high-cognitive level, big idea questions—that are not so often written because they are hard to write and sometimes awkward to grade). This problem of low-cognitive level versus high-cognitive level questions pushes all professors to most frequently create and ask test questions more often about factoids and minutia than any of us really desire. As a result, students want to know which factoids to study because that is the type of questions they will most often encounter from the cadre of professors they experience across campus.
Are you going to ask more high-level questions or more low level questions, or an even mix? One strategy to write a more balanced exam is to create a table of test specifications for what you want your exam to look like BEFORE writing an exam—and before giving students any study sheet that they highly desire. A hypothetical example for a 10-item exam is shown below.
| TOPIC |
Memorization & Naming |
Calculations & Applications |
Evaluating & Synthesizing Ideas |
| -Big Idea #1 |
Q1, Q2 |
|
|
| -Big Idea #2 |
Q3 |
Q4 |
Q5 |
| -Big Idea #3 |
|
Q6, Q7 |
|
| -Big Idea #4 |
Q8 |
|
Q9 |
| -Big Idea #5 |
|
Q10 |
|
The underlying notion here is that you can’t ask about everything, and you don’t want to fall into the unconscious trap of only asking low-cognitive level memorization, vocabulary, and naming questions. Creating a table of specifications allows you to create an exam that is balanced in the ways you want it to be.
This is in stark contrast to sitting down with your notes or the textbook and just starting to write questions—be it multiple choice questions or essay questions. When folks just sit down and start writing questions, the most common result is a long set of memorization and recall questions with very few, if any, higher-level evaluation or synthesizing questions. To avoid this pitfall, design your exam ideal exam before you sit down to write it. Then, after writing a well-designed exam, you’re in a position to provide students with a study sheet that truly represents exactly what it is you want students to do to best position them for a successful final examination performance you can be proud of.
Research tells us that a comprehensive final exam does work in helping your students’ knowledge from your class have durability. This is largely due to a well-documented phenomena in educational psychology known as the “spacing effect.” In other words, students will retain learning longer if they have had to learn and retrieve it several times. If you decide to give students an exam review sheet, then I recommend that you tell them precisely what is actually going to be on the exam, so they can practice mentally retrieving that knowledge, rather that drown in trying to cram an entire term’s worth of lessons in preparation for your exam. Tell them you will ask vocabulary-memory questions related to Big Ideas #x & #y, and that you will ask calculation questions related to Big Idea #z, in our hypothetical example, and you will likely find that they will perform better on your final, even if it is a really tough one, and that they will remember the ideas longer.
Feedback System for Student Voices
Repeatedly, I have tried to tightly connect students having memorable learning experiences and students giving professors high marks on the end-of-course evaluations. The inherent challenge here is that end-of-course evaluations occur at the end of the course, long before you could make any useful course modifications that your students would appreciate. The solution here is to conduct a confidential, mid-term course-evaluation
Without repeating much of long held lamentations of the relative validity, inherent bias by gender and culture, and downright absurdity of many end-of-course evaluation systems, I should acknowledge my sincere belief that most professors really want to do well by the students. I believe that we largely want students to feel like they know more now than they did at the beginning of the class, that they think the subject matter has value and relevance, and that they believe we professors played an important role in helping them learn. At the same time, a reasonable criticism of most student course evaluation systems is that the survey questions usually seem far removed from these goals.
It isn’t that clever scholars studying higher education learning environments specifically haven’t looked at the reliability and validity of student course evaluation systems, for certainly they have. What scholars find is that many of the myths of cajoling students into liking you by dressing informally, telling lots of jokes, bringing donuts, and assigning all A-grades really are myths. In other words, if you were to teach two classes identically, but dress informally, tell more funny stories, deliver pizza, and grade easy in one of your classes, but not your other classes, you’ll find that there really isn’t a significant difference in your end-of-course evaluation scores for the two contrasting, experimental treatments.
So, what characteristics do actually matter and separate out one professor’s end-of-course evaluation scores from another professor? As it turns out, most students don’t carefully read and contemplate each of the individual strongly agree-to-strongly disagree items on the questions’ merit. Rather, interviews with students–both high performing and not so high performing students—reveal that students tend to re-interpret most of the evaluation questions to be about students’ perceptions of just two things:
- Did the professor build and follow an organized pathway designed to help me learn the material?
- Did the professor really care about whether or not I learned the material?
That’s it. Really. All of the other things, like easy tests or premium snacks really don’t impact scores nearly as much as what students perceive about the design of the learning environment and the professor’s posturing. In the end, most of the students desire the same thing most professors hope for: Students want to be different as a result of spending an entire term with you and to not have wasted their time.
This seemingly straightforward result gives you a secret pathway to improve your course evaluation scores, right here in the last stretch of the course. After the mid-term has passed, distribute to your students a simple three question survey during the last ten minutes of a well-attended class.
- Question One: What is one thing that is working really well for you about how this class is structured that we definitely don’t want to get rid of in future course offerings?
- Question Two: What is one thing that you really wish could change about this class that would help you learn better?
- Question Three: Is there anything else that the professor should know?
It is critical that you clearly inform your students that this evaluation information is just for you, and that no one else will see the mid-term results. In this way, you are telling students that they can be honest and that you really care about their learning. After class, find a quiet room with your favorite beverage and start skimming. Make some notes about some themes you see in the notes, both in terms of what is going well, and what is not going so well. You will need these notes for when you de-brief the class at your next class meeting. But, remember, what is most important here is not what the results are; instead, what is most important here is that you let your class know you really care because you asked for their opinion.
At your next class meeting, carry in the stack of surveys and set them down in front of the class. Tell the students that you carefully read every survey and thought deeply about all of their suggestions. This tells the students that not only you cared to ask their opinions about how to help structure the learning environment for them, you cared enough to read their responses (Incidentally, students are largely apathetic about formal campus course evaluation systems too, because they are rightly convinced that those evaluations do little if anything to improve the instruction they themselves experience).
Your task here is to convince students that you care deeply about their learning—deeply enough to find out what is working for them and what isn’t. Tell them which parts of the course design you are going to keep based on their comments. Tell them which parts of the course design you can’t change, because it is too late in the semester. And, most importantly, tell them which parts of the course you can slightly alter in response to their comments so that they can all do well on the final exam. This strategy clearly demonstrates to students that you are on their side, that you are committed to following an organized pathway to help them learn, and that you are leading the class for their benefit, rather than your own ego.
The magic that happens here is when students get to those formal course evaluations at the end of the term, and they start interpreting the items as to measuring the degree to which you gave them an organized pathway and were committed to supporting their learning, the students will report on the fact that you cared deeply enough about their learning to ask. The trade secret here is to give students mid-term course evaluation that demonstrates you are concerned about the same thing the students are: knowing and valuing more than when they started your class those many weeks ago.
Wherever you are on a continuum of courageously and honestly soliciting actionable feedback on your teaching, below are some time-honored ideas that might help you get better data on which to make instructional decisions.
Level 0: End-of-Course Surprise! Some professors never consider the student learning experience during the course and patiently wait – and blindly hope – to receive positive evaluations from the end-of-course student evaluation forms. The relative value of these evaluations has been debated ad nauseam elsewhere, but one thing is for sure, this is the least actionable student feedback on teaching because it happens at the end of a course with students who are likely not deeply invested in providing useful evaluations. This represents the lowest level of interest in getting actionable feedback.
Level 1: How is your attendance? One way to infer how valuable students find your class is to stealthily monitor student attendance. Particularly in large classes, my experience is that professors vastly overestimate actual student attendance rates. Students vote with their feet, and if they can acquire sufficient information to earn a decent grade in your class in ways other than regular class attendance, you might be missing an opportunity to improve the quantity and quality of learning going on during class time. But, in order to see it, you actually have to measure student attendance, which most large enrollment class professors rarely do.
Level 2: Require an Exit Ticket. Professors who want to monitor students’ understanding have long used the strategy of asking students to complete a quick survey at the end of class every couple of weeks. These student-completed, exit-surveys go by many different names: minute paper, muddiest point essay, and exit ticket, are just a few of the names used to describe them. The overarching idea is to occasionally ask students to anonymously answer three questions on a sheet of paper and drop off the paper in a box as they exit the classroom.
My favorite three Exit Ticket questions are:
- What aspect of this class working is working best and should be maintained?
- What aspects of this class should be changed to improve your ability to learn? and
- Is there anything else that the professor needs to know about this class?
In order to solicit useful and honest ideas from students, one can’t require exit tickets too often. If used too frequently, students find them tiring and don’t take them seriously enough. I use exit tickets about twice a month: one time a month about how it is going for a single, specific learning concept I’m trying to teach in the form of a “muddiest point” paper, and one time each month for getting overarching teaching feedback. And, as every master teacher will tell you, be sure you report back to students what you learned from their Exit Ticket and what you are going to do about it, if possible.
Level 3: What’s in the Suggestions Box? A long-standing feedback system for many businesses is a simple Suggestions Box for people to drop off ideas for improvement and, occasionally, an opportunity to praise a particular employee. Today, this can be done quickly, and anonymously, using any of a number of free online survey tools, such as Survey Monkey.
I often set up an online survey with a simple text box for students to provide suggestions and feedback and give students the URL web address (check out mine at: https://www.surveymonkey.com/r/slatersuggestionbox). I do find that I periodically have to remind them that the suggestion box is there, or I get far fewer comments than I would like: Unlike the ominous and reminding wooden suggestion box by the exit, students can forget that an easy electronic pathway exists.
Level 4: Appoint Your Class Leadership Team! For professors who really want to find more ways to intellectually and socially connect with their students, some professors have found soliciting volunteers for a Class Leadership Team, CLT. A CLT is a panel of 6 students who are known to the rest of the class who serve as communication collection points. Students are told that these CLT students are their communication conduit to the professor, and that problems with the way the class are going should be shared with these CLT members. These CLT students can also monitor a physical or online Suggestions Box, and even provide a professor with interpretations of previous semester’s end-of-course evaluation data. Because these CLT students are students, they have tremendous insight into the classroom learning environment. To celebrate and honor the contributions of CLT members, I recommend taking them all out for coffee or lunch every three weeks or so. It costs a few dollars, but the insight and good-intentions are totally worth it. And, making new personal connections with students might mean that your department ends up with a few more declared majors than before.
Effective feedback occurs during the learning, while there is still time for the teacher to act on it. – Jan Chappuis
One thing should be said about soliciting student feedback. Practice shows that you are always going to get more negative comments than positive comments. This is not a strategy to make you feel better about your teaching; instead, this is all about collecting more data so you can make informed decisions. There is no requirement that you act on or respond to every piece of evaluation data. As a general rule, I have to hear something 5 or 6 times before I take it seriously, because some people just find value in criticizing others, even when there is little basis for it. However, if you never ask, you might never find some easy to fix problems in your class or about your teaching style that can be improved before those end-of-course evaluation forms get delivered to your students. If you have an attitude of constantly trying to improve student learning, that will indeed show up positively on your evaluations.
| TRY ITS |
| □ Critically review your syllabus and be sure that daily course activities are listed _AND_ that a weekly student to-do checklist is included
□ Be sure your syllabus helps students always know where they stand grade-wise and how their grades are determined
□ The critical fail is for your exams to not be obviously aligned with what happens in class
□ Find ways for students to share their feedback so you can improve their learning experience and demonstrate your vested interest in their success |
Earlier posts in this series are:
An anticipated upcoming post in this innovative college astronomy series is:
- What’s Your Astronomy Teaching Paradigm?
Astronomy educators and outreach enthusiasts have all sorts of avenues to excite learners about the wonders of the universe—and the K-12 school planetarium is no exception. So, just what does a K-12 school planetarium director do anyway? 
Special skills required:
When you want students telling other students about what it is like to be a student in your astronomy class, what do you want them to say? Do you really want them to say, “that professor was a gatekeeper, and kept me from graduating?” Do you hope that they will say, “the professor was a great lecturer, but the lectures didn’t help me pass the exams?” Or, “that class was ridiculously hard: I hated every minute of it.”
Most of us hope that our students say good things about our course both to other students and—more importantly—on our end-of-course evaluations. When students say positive things about your course to other students, this helps increase your enrollment because students do act upon instructor-recommendations from one another when selecting their courses. At the same time, when students say positive things about your on your end-of-course evaluations, this helps you get high marks on your performance and promotion reviews.
Instead, what I want you to do is to be highly successful at teaching students who enter your classroom already convinced they don’t love astronomy. If you organize your class for these hard-to-reach students, nearly everyone wins—even those students who enrolled in your class correctly thinking astronomy is awesome.
YEAHBUT my students won’t do anything that isn’t graded
(1) select an article and describe why it is directly relevant;
To mitigate for this, I suggest that you ask students to “do something” at the telescope. If you have enough time, the absolute best thing to do is to have students sketch what they see in the telescope. Sketching Jupiter’s belts or the Moon’s craters makes students allocate just enough extra-time at the eyepiece to be able to have a meaningful learning experience.
What sort of specific tasks should students be doing during a field trip? The time tested approach here is to have students write, sketch, and reflect on things that they see and experience, usually through a worksheet. The standard time-tested method here is to provide students with a task list of things to do and sketch while they are visiting the facility. But, all task sheets are not created equal. Let’s compare and contrast two, illustrative field trip task sheets—the Scavenger Hunt and the Inquiry Guide.
Although many of our colleagues recommend using 12 in. x 12 in., I personally like 24 in. x 24 in. because several students can write on it at once. You can keep a stack in the classroom or move them around easily which an inexpensive dolly. And, if they wear quickly, these boards can be replaced at a very low cost. You’ll also need a box of dry erase markers (or task students to bring their own) and a bunch of cheap black socks—available at your nearby dollar-store—to use as erasers.
During class classroom activities have been around for a long time. The 
Recently, work by 
What is particularly unique about BFS laboratory learning experiences is the recognition that teaching students to ask scientifically fruitful questions is by far the most difficult aspect. In response, the BFS research experiences teach students to first create and defend evidence-based conclusions from a given research question and given data. Once students have mastered this, students are then taught to devise strategies to pursue data that can be used in an evidenced-based conclusion for a new research question, which is also provided for them. Only when students have had considerable experience designing observations and defending conclusions from a number of research questions, are students then taught to create fruitful research questions—now that they have considerable experience doing the processes of scientific inquiry in astronomy.
Let’s consider an illustrative example. Suppose a BFS instructional sequence in scientific inquiry is to ask students to use an online database of solar system objects showing the planet and moon positions and their motions to pursue a series of investigations.
But what do you do with this? We propose the answer is to have students participate in a SCIENCE SYMPOSIA. Allow us to explain.
With all that preamble, what we are trying to say is that one strategy to radically alter the commonly used end-of-term essay assignment and instead host a student-lead mini-science conference. At the beginning of the course, students can be assigned the task of completing a scientific investigation of their choosing and create an illustrated poster presentation, much like is done at professional science conferences. Then, you spend the entire term preparing them for doing this, with lots and lots of practice.
if the PPTs are available elsewhere. We have to agree with the students on this one; if the only reason students have to come to class is to get information to memorize from the PPT, why go to class at all? The remaining chapters in the book are written specifically to counteract this notion: Your class time should be so well organized and carefully designed to be so incredibly valuable that your students wouldn’t imagine missing it in their wildest dreams. If that’s not motivation enough to keep reading this book, we don’t know what would be!
CAPER Astronomy Faculty Lounge 