End-of-Term Evaluations

Well-designed courses create a structured but open-ended environment that encourages students to take primary responsibility for their learning, motivates and facilitates their continued engagement, and offers them ample opportunities for assessment of and reflection on their progress.

Reviewing end-of-term evaluations may be an opportunity to improve techniques in these areas. Evaluations also signal to students what the institution thinks are the most important features of learning and prompt them to reflect on their experiences in your course.

Dartmouth College has an online system for end-of-term course evaluations that is available through Banner. We have organized this brief guide following the structure of the report you receive. For instructions on how to access your reports and a look at the form as the students see it, visit the Registrar’s course assessment guide for faculty.

Before you look over your most recent set of course evaluations, consider studying them in the company of a trusted colleague. Being evaluated, especially by our students, can feel unsettling. The DCAL director and associate directors stand ready to help you read your evaluations, interpret them and put them to good use. Call or email DCAL to set up an appointment.

Bias

When students evaluate courses and the faculty who teach them, bias sometimes plays a role, especially when instructors are perceived as belonging to a population under-represented in the faculty ranks. Such biases may reflect attitudes that stereotype race, gender, religion, ethnic origin and age. For instance, research shows that gender expectations (Sprague and Massoni, 2005), racial bias (Hendrix, 1998;Smith and Anderson, 2005), and even aesthetic preferences (Hamermesh and Parker, 2005) shape some students’ evaluations of their college professors. The influence of bias may vary depending on factors such as class size (Hameed, 2014;Martin, 2013).

Students may be unaware of the biases that affect them, or they may express them quite deliberately. In either case, faculty members who suspect that course evaluations reflect such bias should not hesitate to speak with their department chair, their dean, or DCAL staff.

Navigating the Online “Course Assessment Summary”

Once you have logged onto the Dartmouth College Student Information System (SIS or Banner), click on “Dartmouth Faculty and Advisor Main Menu” and then on “Course Assessment Reports.” You will find yourself on the “Course Level Reports” page. Click on “Course List” to access a list of courses by term; the most recent term you taught will appear first; you may select a different term by using the drop-down menu near the top of the page. Next you can select “Single Course Report” to see an aggregation of all the student responses for that course on a single page. It is also very helpful to look at each individual set of responses, that is, each student’s set of responses to the assessment questionnaire. For that, click on “Individual Response Reports.” This allows you to browse through each student’s whole set of responses, one at a time. This can help you interpret the responses; for example, a student who reports spending fewer than 5 hours each week on your course may also have indicated a lack of intellectual engagement. Returning to the “Main Menu,” you may also choose to see various comparison reports and even college-wide reports.

Mean and Median

Sometimes you glance first at the median and mean score for each question on the “Single Course Report.” It’s important to remember the median and mean tell us different information. The median is the middle score of all the scores in numerical order. The mean is the average score; if you added up all the response scores (remember at Dartmouth 1 is the highest and 5 is the lowest) and then divided by the number of responses for that item, you will get the mean score. (“N/A” does not count as a scored response.). Many people focus on the mean, but the median is equally important and you want to consider both together.

Averages do not always tell a useful story without considering the median. Imagine an extreme case in which 24 out of 30 students rated a course “Excellent” and 6 rated it “Poor.” The resulting average or mean would be 1.8 or “Very Good.” The 6 dissatisfied students severely affect the average. By computing the median, however, you will get another picture. To compute the median, arrange all the scores from highest to lowest and select the middle score. (If there is no middle score, then average the two in the middle.) In our extreme case, then, the median would be 1 or “Excellent.” So the average tells one story, and the median tells another. You want to know that six students were pretty dissatisfied and think of ways to address that, but you also want to see that for most of the students, the course was “Excellent.” Because the summary reports both mean and median you get a look at both stories.

I. Student Background

This first section of the evaluation response contains useful information about the students who completed evaluations for your course. This information may help you determine whether the course attracted the students you expected. Were they mostly sophomores, or seniors, or a mix? Were most of them trying to satisfy a requirement? These factors may influence how you understand the evaluations that follow and they may even prompt you to think about planning for different constituencies.

Students who report high attendance rates and ten or more hours per week spent on your course were probably highly engaged by your course. The grade expectations they report may help you compare your sense of their achievement to theirs. The grades LP (low pass), P (pass) and HP (high pass) are used in graduate level courses. Students who take the course for CT (credit) only or under an NRO (non-recording option) agreement may have significantly different perspectives on the course than those who took it for a grade. All of this information can be useful in future course planning and adjustments.

II. Course Design and Effectiveness

1. I think the overall quality of the course was _______. High-quality learning experiences have as much to do with well-designed syllabi, learning exercises, and assessments as with an instructor's performance in class. Consider that most student learning in your course will take place outside of class periods (McKeachie, 2002); therefore, this question speaks to students' perceptions of the quality of assignments and evaluations, in addition to their in-class experiences. Students generally give high marks for overall quality when they feel that they understood very clearly the objectives, goals or outcomes the instructor intended the course to promote and these coincided with their own reasons for taking the course. To avoid mismatched sets of expectations which may inhibit learning, it is a good idea to plan your course following a procedure called "backwards design" (Wiggins and McTighe, 2005). This involves articulating the outcomes you want (What do you want your students to know and be able to do with that knowledge by the end of your course?), and then deciding how you will know when the students have achieved these outcomes. A clear sense of these learning goals facilitates designing assessments and selecting assignments and exercises to achieve these goals.

State your desired outcomes at the top of your syllabus and take a sentence or two for each assignment to explain how it furthers the goals you have set for learning. If you use a Canvas site for the course, give the students enrolled in the course a heads-up e-mail a week or more before the first class meeting and invite them to browse the syllabus and assignments. It's also a good idea to have them complete a short pre-course survey to prompt them to reflect on their reasons for signing up for your course and so that you can know more about what they know, what they expect from the course and how they think they learn best. DCAL and the instructional designers from Educational Technology can help you set this up quickly.

2. I learned a lot in this course. Students strongly agree with this statement when they feel that the high expectations you set out for them matched their own reasons for taking the course and when they have had plenty of opportunities to assess their success and reflect on that assessment. Ken Bain (2006) writes that people learn best and most deeply when "they can try, fail, receive feedback, and try again before anyone makes a judgment of their work." This means that learning activities like practice tests and quizzes that students can take over and over and opportunities to rewrite essays for full marks help students learn.

3. I put a great deal of effort into the course. What motivates students to put more effort into a course? Alexander Astin's Theory of Involvement essentially states that "students learn by becoming involved" (1985). This theory builds off of the notion of student invested time-on-task. One of Astin's five posits is that "the amount of learning or development is directly proportional to the quality and quantity of involvement" (pp. 135-136). Pascarella and Terenzini (2005) reflected on Astin's theory: "the student, however, plays the lead role inasmuch as change is likely to occur only to the extent that the student capitalizes on opportunities and becomes involved, actively exploiting the opportunities to change or grow that the environment presents. Thus, development or change is not merely the consequence of college's impact on a student but rather a function of the quality of student effort or involvement with the resources provided by the instructor" (pp. 53-54).

Students respond especially well when they believe their teacher is genuinely invested in their success. Autonomy, independence and relevance of material are also highly motivating for students. Assignments and class time should be optimized to initiate "tuning" moments (e.g., opportunities where the faculty member can provide feedback to the student on their performance in class). Early and frequent feedback from the professor is key, and if it contributes to students' recognizing they are making progress, this can be highly motivating (Svinicki, 2004). Faculty members trying to encourage greater student involvement may consider including more formative assessments and active learning in their courses.

4. I was intellectually engaged in the course. John Dewey once said,"We don't learn from experience; we learn from reflecting on experience." Students feel most engaged in learning when they have lots of opportunities and prompts to reflect on the content of your course. Instead of asking students if they have questions, assume they do and ask them what questions they have. Ask them to formulate new questions and re-formulate old ones. Provide plenty of motivation for reflection and self-assessment and allow time for it to be done well. If your course is a large lecture course keeping students actively engaged can be particularly challenging, but many teachers do this successfully every term. The people at DCAL can help you meet those challenges efficiently.

5. The objectives of the course were clear to me. Your expectations and your students" expectations will not always mesh, but that's all the more reason that you take time to be explicit and allow your students to tell you about their expectations. They do not have to match exactly, but it helps if everyone acknowledges where expectations and goals diverge. State your desired outcomes at the top of your syllabus and take a sentence or two for each assignment to explain how it furthers the goals you have set for learning in your course. If you use a Canvas site for the course, give the students enrolled in the course a heads-up e-mail a week or more before the first class meeting and invite them to browse the syllabus and assignments. It's also a good idea to have them complete a short pre-course survey (links to DCAL resource) to prompt them to reflect on their reasons for signing up for your course and so that you can know more about what they know, what they expect from the course and how they think they learn best. The DCAL director and associate directors can help you set this up.

6. I found the course to be well organized. Students may not be in a position to see how well you have organized a course; nevertheless, even a well-organized course can appear disorganized to an inexperienced student and it helps if we keep our students informed about why we do the things we do. When we tell students how our choices of readings, assignments and assessments relate to the learning goals of the course, they are more likely to see how a course is organized and why.

7. The assignments reinforced my understanding of the course material. Whether the assignment is a paper, a problem set or a lab report, we should make explicit what learning goals we expect the assignment to help students achieve. Is it about memorizing details? mastering a disciplinary discourse? learning methods of research or procedures? Generally, an assignment has more than one learning goal in mind. Make these explicit on the assignment sheet or in the syllabus. Sometimes it makes sense to include a set of rubrics by which you will assess how well the assignment has been done.

8-10. Free text responses. For most of us, these are the most difficult parts to understand in a course evaluation, because students sometimes offer contradictory responses and suggestions. These contradictions tempt us to think such evaluations are meaningless, but they are not. One student's favorite reading may well seem dispensable to another; that actually makes sense because they are different!

Here are some helpful tips for reading these comments: 

Read and compare comments in one category at a time. Don't pay much attention to contradictory comments on your first reading, but be sure to take special note of any comment or suggestion that appears more than once.

Because students write evaluations anonymously, students may write comments that appear negative or sarcastic. Such comments may be motivated by circumstances completely unrelated to your course. That said, such comments appear less frequently when students feel they understood an instructor's expectations very well and also believe the instructor understood and respected their reasons for being in the course. Grades affect student responses less when grading rubrics have been explicit and grading is perceived as consistent and reasonable. Remember, feedback is actually more important than grades to our best students, so giving an A with little or no constructive feedback can generate a negative comment.

Difficult as it may be, pay close attention to any criticism that appears more than once, even if it is a minority opinion and many other positive comments contradict it. There may be a small group of students regularly attracted to your course who could benefit from another approach or a different kind of assessment.

Because this is often the hardest part of the evaluations to absorb, consider reading these together with the director or an associate director at DCAL. You may be assured of perfect confidentiality.

III. Faculty

1. I think the overall effectiveness of the teaching was ____. Whether or not you believe students are good judges of your effectiveness as a teacher, what they think on this score matters and deserves our attention. We should all keep in mind that what one student finds effective for her learning style may not suit another, so it is almost impossible to get consistently high marks on this, nor should we expect that. In general, students give high marks for effectiveness when they believe that we are invested in their success and when the gap between our expectations about the coursework and theirs is as narrow as possible. So frequent assessments, opportunities to try again, patient feedback, and clear articulation of your expectations can all help.

2. The professor set high standards. Contrary to popular mythology, students like to be challenged. The more we can fashion our courses as invitations to scholarly work in our disciplines, the better they respond. Even where that is not possible, students respond well to high expectations as long as there is plenty of support for their achievement. Let them know on day one and on your course website about programs designed to support their learning, such as the Academic Skills Center, RWIT, and Student Accessibility Services.

3. The professor explained central concepts clearly. We have lived and breathed our disciplinary discourse for at least a decade. Students have not. What appears intuitive to us may appear pretty opaque to them. It's a good idea to allow at least twice as much time for them to absorb material as we would expect of ourselves. It's also a good idea to explain key concepts several different ways with several different examples. After one explanation, ask students to reformulate it in writing. Then offer another example and ask for another reformulation that takes the new example into account.

4. The professor challenged me to think critically about the course material. Critical thinking looks a bit different in every discipline, but one trait is common to all—the formulation and re-formulation of questions. Students who can frame new questions about old topics demonstrate that they have done more than "master material" — they have begun to engage critically with the subject. We should reward good questions at least as much as good answers, maybe more. To repeat a point made above, rather than asking students if they have questions, we should assume they do and ask them to articulate them for us and their peers.

5. The professor was available for consultation outside of class. Students prize whatever one-on-one time you can offer them. They selected Dartmouth because we promised them more of this than other schools do. More important, such consultations promote deeper more engaged learning. Regular office hours can satisfy this, but consider lunch and coffee appointments as well. Like anyone else, students respond better to direct invitations.

6-8. Free text responses. See above in section II, questions 8-10.

IV. Discussion/Lab/Drill Integration

1. The discussion section/lab/drill section was well integrated into the course structure. Lab instructors should know exactly what learning goals you have set for the course, how the readings and assignments support those goals and conduct lab sessions accordingly. Much the same is true of discussion leaders and drill instructors. If we take time to discuss our goals and expectations with our co-instructors and review with them our criteria for assessment, students will perceive these activities as well integrated learning experiences.

Astin, A.W. (1985). Involvement: The cornerstone of excellence. Change: The Magazine of Higher Learning, 17(4), 35-39.

Bain, K. (2004). What the best college teachers do. Cambridge, MA: Harvard University Press. (Available in DCAL's library)

Benton, S. (2012). Students ratings of teaching: A summary of research and literature. IDEA Paper #50.

Hameed, F., Ali, A., Hameed, A. et al. (2014). Teacher evaluation: The role of gender. Quality & Quantity, 49(5), 1779-1789.

Hamermesh, D., & Parker, A. (2005). Beauty in the classroom: Instructors' pulchritude and putative pedagogical productivity. Economics of Education Review, 24(4), 369-376.

Hendrix, K. (1998). Student perceptions of the influence of race on professor credibility. Journal of Black Studies, 28(6), 738-763.

Martin, L. (2013, August). Gender, teaching evaluations, and professional success in political science. In the American Political Science Annual Meeting, Chicago, IL.

McKeachie, W. (2002). McKeachie's teaching tips: Strategies, research, and theory for college and university teachers (11th ed.). Boston: Houghton Mifflin. (Available in DCAL's library)

Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: a third decade of research (3rd ed.). San Francisco, CA: Jossey-Bass.

Smith, G. & and Anderson, K. (2005). Students' ratings of professors: The teaching style contingency for Latino/a professors. Journal of Latinos and Education, 4(2), 115-136.

Sprague, J. & Massoni, K. (2005). Student evaluations and gendered expectations: What we can't count can hurt us. Sex Roles, 53(11/12), 779-793.

Svinicki, M. (2004). Learning and motivation in the postsecondary classroom. Bolton, MA: Anker Publishing Company.

Wiggins, G. & McTighe, J. (2005). Understanding by design (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development. (Available in DCAL's library)