Dartmouth Course Assessment Questions
I. Student Background
This first section of the evaluation response contains useful information about the students who completed evaluations for your course. This information may help you determine whether the course attracted the students you expected. Were they mostly sophomores, or seniors, or a mix? Were most of them trying to satisfy a requirement? These factors may influence how you understand the evaluations that follow and they may even prompt you to think about planning for different constituencies.
Students who report high attendance rates and ten or more hours per week spent on your course were probably highly engaged by your course. The grade expectations they report may help you compare your sense of their achievement to theirs. The grades LP (low pass), P (pass) and HP (high pass) are used in graduate level courses. Students who take the course for CT (credit) only or under an NRO (non-recording option) agreement may have significantly different perspectives on the course than those who took it for a grade. All of this information can be useful in future course planning and adjustments.
II. Course Design and Effectiveness
1. I think the overall quality of the course was _______. High-quality learning experiences have as much to do with well-designed syllabi, learning exercises, and assessments as with an instructor's performance in class. Consider that most student learning in your course will take place outside of class periods (McKeachie, 2002); therefore, this question speaks to students' perceptions of the quality of assignments and evaluations, in addition to their in-class experiences. Students generally give high marks for overall quality when they feel that they understood very clearly the objectives, goals or outcomes the instructor intended the course to promote and these coincided with their own reasons for taking the course. To avoid mismatched sets of expectations which may inhibit learning, it is a good idea to plan your course following a procedure called "backwards design" (Wiggins and McTighe, 2005). This involves articulating the outcomes you want (What do you want your students to know and be able to do with that knowledge by the end of your course?), and then deciding how you will know when the students have achieved these outcomes. A clear sense of these learning goals facilitates designing assessments and selecting assignments and exercises to achieve these goals.
State your desired outcomes at the top of your syllabus and take a sentence or two for each assignment to explain how it furthers the goals you have set for learning. If you use a Canvas site for the course, give the students enrolled in the course a heads-up e-mail a week or more before the first class meeting and invite them to browse the syllabus and assignments. It's also a good idea to have them complete a short pre-course survey to prompt them to reflect on their reasons for signing up for your course and so that you can know more about what they know, what they expect from the course and how they think they learn best. DCAL and the instructional designers from Educational Technology can help you set this up quickly.
2. I learned a lot in this course. Students strongly agree with this statement when they feel that the high expectations you set out for them matched their own reasons for taking the course and when they have had plenty of opportunities to assess their success and reflect on that assessment. Ken Bain (2006) writes that people learn best and most deeply when "they can try, fail, receive feedback, and try again before anyone makes a judgment of their work." This means that learning activities like practice tests and quizzes that students can take over and over and opportunities to rewrite essays for full marks help students learn.
3. I put a great deal of effort into the course. What motivates students to put more effort into a course? Alexander Astin's Theory of Involvement essentially states that "students learn by becoming involved" (1985). This theory builds off of the notion of student invested time-on-task. One of Astin's five posits is that "the amount of learning or development is directly proportional to the quality and quantity of involvement" (pp. 135-136). Pascarella and Terenzini (2005) reflected on Astin's theory: "the student, however, plays the lead role inasmuch as change is likely to occur only to the extent that the student capitalizes on opportunities and becomes involved, actively exploiting the opportunities to change or grow that the environment presents. Thus, development or change is not merely the consequence of college's impact on a student but rather a function of the quality of student effort or involvement with the resources provided by the instructor" (pp. 53-54).
Students respond especially well when they believe their teacher is genuinely invested in their success. Autonomy, independence and relevance of material are also highly motivating for students. Assignments and class time should be optimized to initiate "tuning" moments (e.g., opportunities where the faculty member can provide feedback to the student on their performance in class). Early and frequent feedback from the professor is key, and if it contributes to students' recognizing they are making progress, this can be highly motivating (Svinicki, 2004). Faculty members trying to encourage greater student involvement may consider including more formative assessments and active learning in their courses.
4. I was intellectually engaged in the course. John Dewey once said,"We don't learn from experience; we learn from reflecting on experience." Students feel most engaged in learning when they have lots of opportunities and prompts to reflect on the content of your course. Instead of asking students if they have questions, assume they do and ask them what questions they have. Ask them to formulate new questions and re-formulate old ones. Provide plenty of motivation for reflection and self-assessment and allow time for it to be done well. If your course is a large lecture course keeping students actively engaged can be particularly challenging, but many teachers do this successfully every term. The people at DCAL can help you meet those challenges efficiently.
5. The objectives of the course were clear to me. Your expectations and your students" expectations will not always mesh, but that's all the more reason that you take time to be explicit and allow your students to tell you about their expectations. They do not have to match exactly, but it helps if everyone acknowledges where expectations and goals diverge. State your desired outcomes at the top of your syllabus and take a sentence or two for each assignment to explain how it furthers the goals you have set for learning in your course. If you use a Canvas site for the course, give the students enrolled in the course a heads-up e-mail a week or more before the first class meeting and invite them to browse the syllabus and assignments. It's also a good idea to have them complete a short pre-course survey (links to DCAL resource) to prompt them to reflect on their reasons for signing up for your course and so that you can know more about what they know, what they expect from the course and how they think they learn best. The DCAL director and associate directors can help you set this up.
6. I found the course to be well organized. Students may not be in a position to see how well you have organized a course; nevertheless, even a well-organized course can appear disorganized to an inexperienced student and it helps if we keep our students informed about why we do the things we do. When we tell students how our choices of readings, assignments and assessments relate to the learning goals of the course, they are more likely to see how a course is organized and why.
7. The assignments reinforced my understanding of the course material. Whether the assignment is a paper, a problem set or a lab report, we should make explicit what learning goals we expect the assignment to help students achieve. Is it about memorizing details? mastering a disciplinary discourse? learning methods of research or procedures? Generally, an assignment has more than one learning goal in mind. Make these explicit on the assignment sheet or in the syllabus. Sometimes it makes sense to include a set of rubrics by which you will assess how well the assignment has been done.
8-10. Free text responses. For most of us, these are the most difficult parts to understand in a course evaluation, because students sometimes offer contradictory responses and suggestions. These contradictions tempt us to think such evaluations are meaningless, but they are not. One student's favorite reading may well seem dispensable to another; that actually makes sense because they are different!
Here are some helpful tips for reading these comments:
Read and compare comments in one category at a time. Don't pay much attention to contradictory comments on your first reading, but be sure to take special note of any comment or suggestion that appears more than once.
Because students write evaluations anonymously, students may write comments that appear negative or sarcastic. Such comments may be motivated by circumstances completely unrelated to your course. That said, such comments appear less frequently when students feel they understood an instructor's expectations very well and also believe the instructor understood and respected their reasons for being in the course. Grades affect student responses less when grading rubrics have been explicit and grading is perceived as consistent and reasonable. Remember, feedback is actually more important than grades to our best students, so giving an A with little or no constructive feedback can generate a negative comment.
Difficult as it may be, pay close attention to any criticism that appears more than once, even if it is a minority opinion and many other positive comments contradict it. There may be a small group of students regularly attracted to your course who could benefit from another approach or a different kind of assessment.
Because this is often the hardest part of the evaluations to absorb, consider reading these together with the director or an associate director at DCAL. You may be assured of perfect confidentiality.
1. I think the overall effectiveness of the teaching was ____. Whether or not you believe students are good judges of your effectiveness as a teacher, what they think on this score matters and deserves our attention. We should all keep in mind that what one student finds effective for her learning style may not suit another, so it is almost impossible to get consistently high marks on this, nor should we expect that. In general, students give high marks for effectiveness when they believe that we are invested in their success and when the gap between our expectations about the coursework and theirs is as narrow as possible. So frequent assessments, opportunities to try again, patient feedback, and clear articulation of your expectations can all help.
2. The professor set high standards. Contrary to popular mythology, students like to be challenged. The more we can fashion our courses as invitations to scholarly work in our disciplines, the better they respond. Even where that is not possible, students respond well to high expectations as long as there is plenty of support for their achievement. Let them know on day one and on your course website about programs designed to support their learning, such as the Academic Skills Center, RWIT, and Student Accessibility Services.
3. The professor explained central concepts clearly. We have lived and breathed our disciplinary discourse for at least a decade. Students have not. What appears intuitive to us may appear pretty opaque to them. It's a good idea to allow at least twice as much time for them to absorb material as we would expect of ourselves. It's also a good idea to explain key concepts several different ways with several different examples. After one explanation, ask students to reformulate it in writing. Then offer another example and ask for another reformulation that takes the new example into account.
4. The professor challenged me to think critically about the course material. Critical thinking looks a bit different in every discipline, but one trait is common to all—the formulation and re-formulation of questions. Students who can frame new questions about old topics demonstrate that they have done more than "master material" — they have begun to engage critically with the subject. We should reward good questions at least as much as good answers, maybe more. To repeat a point made above, rather than asking students if they have questions, we should assume they do and ask them to articulate them for us and their peers.
5. The professor was available for consultation outside of class. Students prize whatever one-on-one time you can offer them. They selected Dartmouth because we promised them more of this than other schools do. More important, such consultations promote deeper more engaged learning. Regular office hours can satisfy this, but consider lunch and coffee appointments as well. Like anyone else, students respond better to direct invitations.
6-8. Free text responses. See above in section II, questions 8-10.
IV. Discussion/Lab/Drill Integration
1. The discussion section/lab/drill section was well integrated into the course structure. Lab instructors should know exactly what learning goals you have set for the course, how the readings and assignments support those goals and conduct lab sessions accordingly. Much the same is true of discussion leaders and drill instructors. If we take time to discuss our goals and expectations with our co-instructors and review with them our criteria for assessment, students will perceive these activities as well integrated learning experiences.