Methods for Continuous Improvement

Utilizing Student Evaluations and Self-Evaluations for Teaching Improvement

Continuous improvement, also known as course refinement or instructor refinement, refers to the ‘continuous’ process of instructors evaluating and improving upon teaching practices based on student evaluations or self-reflection (Taylor et al., 2020). Evaluations are a tool, typically taking the form of a survey or discussion given to students, used to improve the instructor’s teaching and gauge an idea of how students are experiencing the class (Taylor et al., 2020).

Student evaluations benefit students, faculty, and administration (Taylor et al., 2020). For students and faculty, student evaluations serve as an opportunity to open a dialogue between instructors and their students about what they do that works, identify opportunities for growth or pivot points in the instructor’s teaching, improve student experience, and establish a connection of respect between both parties that allows students to feel seen, heard, and valued (Taylor et al., 2020; Blash et al., 2020). End-of-semester student evaluations also serve as an administrative tool to assess faculty member’s performance in their role (Diaz et al., 2022). The role of instructor universally requires a process of observation, reflection, and adaptation to identify opportunities for improvement and to foster a more inclusive teaching practice (The University of Washington, n.d.).

Learning how an instructor can improve their teaching does not have to come from student voices alone (Cole et al., 2022). Critical self-reflection and evaluating one's own teaching are fundamental practices for instructors (Cole et al., 2022). Therefore, instructors grant themselves a great advantage by understanding tools and strategies available to them, supported by peer-reviewed research, to enhance this reflective process through student feedback.

End-of-Semester Evaluations

Instructors and students are likely familiar with end-of-semester evaluations. These are typically questionnaires with mixed quantitative and qualitative items that students complete at the end of the semester. The questionnaires invite student perspective on instruction and course materials. The results are shared with instructors by administrators with the goal of measuring performance or making improvements to teaching or course design. In some instances, they may be taken into consideration for “high stakes” decisions such as tenure or course offerings (Watermark Insights, 2022; Diaz et al., 2022).

There are benefits and drawbacks to end-of-semester evaluations. While end-of-semester evaluations are typically a ubiquitous staple in American higher education classrooms, instructors should understand the ways in which it is and is not helpful for their growth as instructors (Diaz et al., 2022).

Benefits of end-of-semester evaluations:

• Provides information on course delivery and quality (Diaz et al., 2022).
• Utilized by administration to help determine tenure and promotion (Diaz et al., 2022).

Drawbacks of end-of-semester evaluations:

• Does not benefit students who provided the feedback (Marx, 2019).
• Not very effective for measuring teaching effectiveness and student learning (Diaz et al., 2022; Marx, 2019).
• Standardized across disciplines; collects generalized data (Marx, 2019; Frick et al., 2010).
• Administrative use of end-of-semester evaluations does not consider student biases or response rates (Diaz et al., 2022).

The efficacy of end-of-semester evaluations is a highly debated topic among instructors. In a study by Diaz et al. (2022) which examined how faculty utilize end-of-course evaluations, almost all participants had reported using end-of-course evaluations to inform their approach to future courses and their general teaching practices (Diaz et al., 2022, p. 291). However, more than 25% of faculty participants said that they received no valuable feedback on assignment and project quality and felt that they gathered no useful information from closed ended (quantitative) survey items (Diaz et al., 2022). A number of participants also felt skepticism over their value due to student biases and low response rates (Diaz et al., 2022). Additionally, they do not measure student learning or outcomes, and  show no statistical correlation with student learning (Diaz et al., 2022; Boring et al., 2016; Uttl et al., 2017). In sum, end-of-semester evaluation feedback can provide some useful feedback for instructor’s teaching practices and course design, but overall, does not provide a comprehensive picture of their teaching or student experiences and outcomes, making it a good, but not a great, method to collect and utilize student feedback to enhance continuous improvement.

Mid-Semester Evaluations

Mid-semester evaluations prioritize student feedback for reflection and teaching growth rather than just assessment (Marx et al., 2019). They can cut to student feedback that is more in-depth and specifically to instructor’s teaching practices rather than overview the entire course in general terms (Marx et al., 2019). Additionally, mid-semester evaluations can benefit students just as much as the instructor since course changes can be made mid-semester (Marx et al., 2019; Taylor et al., 2020). This also communicates to students that their perspectives and active participation in the course are valued and respected (Marx et al., 2019; Taylor et al., 2020). Both parties benefit from an open dialogue on feedback given, making mid-semester evaluations an effective way to improve one’s teaching and connect with students (Marx et al., 2019; Taylor et al., 2020). The following strategies can be used to implement mid-semester evaluations in the higher education classroom.

Small Group Instructional Diagnosis (SGID)

Small Group Instructional Diagnosis (SGID) is a structured interview process between students and a facilitator other than the faculty member, for example, someone from the Teaching Center or similar departments in the university (Blash et al., 2018). SGID occurs half-way through the semester and allows students to identify factors that contribute to or detract from their learning and recommend ways that the instructor could improve their teaching in the course moving forward prior to the end of the course (Blash et al., 2018).

How Small Group Instructional Diagnosis (SGID) Works:

• The instructor informs students about the upcoming review and emphasizes that it is a voluntary but appreciated activity that will allow students to offer feedback on the instructor and the course (Blash et al., 2018).
• The facilitator comes into class and introduces themselves, stating that the faculty member wants to conduct this evaluation because they care about improving their course and student experiences (Blash et al., 2018).
• The facilitator distributes a form asking questions about what is and is not working in the class, and what changes they would like (Blash et al., 2018). The students fill out the form with qualitative responses.
• Students are then arranged into groups where they collectively answer the questions on the original form (Blash et al., 2018).
• Each group shares their responses with everyone, synthesizing important points of feedback to return to the instructor (Blash et al., 2018).
• The facilitator makes note of the feedback and interprets it into direct feedback for the faculty member, who then may discuss the anonymously collected feedback with the class and determine how to move forward and what changes they will implement or not (Blash et al., 2022).  

Why Use SGID?

• Allows for a more fluid and collaborative dialogue between students and instructors that a simple survey or questionnaire cannot provide (Taylor et al., 2020).
• Shows students their feedback is valued and that they play a tangible role in their learning (Taylor et al., 2020).
• Connects instructors with peers or faculty developers (Taylor et al., 2020).
• Encourages active learning for instructors and students alike (Taylor et al., 2020).
• Instructor has greater control over the content of the evaluation and what areas of their teaching or course delivery they can garner feedback on (Svendby, 2021).

‘Stop, Start, Continue’ (SSC) Method

The ‘Stop, Start, Continue’ method of collecting mid-semester feedback, also known as the “Keep, Quit, Start” method – asks students on what instructors should stop doing (stop/quit), should add (start), and keep doing (continue/keep) in regard to their teaching practices (Cunningham & White, 2022). A study by Cunningham & White (2022) showed that instructors preferred student feedback that was qualitative in nature (written out) as opposed to quantitative (simple survey answers). Qualitative responses require thematic analysis that is more involved than simply reviewing survey data and numbers, however, can garner more valuable and specific feedback for instructors and allow for student voices to more accurately be heard (Cunningham & White, 2022). This three-item qualitative questionnaire can be a highly efficient and straight-forward way to collect mid-semester feedback from students

How to Utilize a SSC Evaluation:

• Collect responses in a way that maintains student anonymity (Boston University, n.d.). This can be done through various online survey tools or by leaving the classroom while students fill in a paper-questionnaire and leaving their responses in a pile on the instructor’s desk so they do not know who wrote which response.
• Consider organizing feedback into a table such as the one shown below. Under each column, ‘Start’, ‘Stop’, and ‘Continue’, list, in order of frequency, changes students have recommended their instructor make for each category. Listing feedback items in order of frequency can help highlight priority items. Please see this table, adapted from The Boston University Center for Teaching and Learning:

(Boston University, n.d.)

• After collecting responses, instructors critically reflect on feedback and determine what is possible or helpful to change and be ready to discuss the overall feedback with the class (Boston University, n.d.).
• Instructors review feedback with class during the following class period and discuss response to the feedback, what they are willing to change and what will remain the same (Boston University, n.d.). Instructors should be sure to open a dialogue with students about how to proceed with the feedback given (Boston University, n.d.).

Why Utilize the SSC method?

The SSC method can be a straightforward way to cut to the core of what students want, do not want, and currently enjoy about the instructor’s teaching and course delivery and focuses on potential action items. This method does not go in depth on student perceptions of their instructor’s teaching or a deeper description of their experiences; however, it is great for gauging what students need from the course.

There are numerous methods supported by peer-reviewed data that instructors can utilize in their classroom to collect valuable and insightful mid-semester feedback that benefits both the instructor and their students. Mid-semester evaluations put the control in the instructor’s hands and create opportunities for dialogue with students, involving them in the teaching and learning process. While each method described has its own unique strengths, pitfalls, and processes – below are some general tips that can assist instructors in collecting mid-semester evaluations regardless of what method they employ.

Tips for Utilizing Mid-Semester Evaluations in Your Classroom

• Collect data anonymously (Svendby, 2021). This may seem obvious, but it is so important that it bares mention. The instructor/student power dynamics are imbalanced, and students face potential retaliation for perceived negative feedback which can put them at risk and lead them to feel unsafe to provide honest feedback. By allowing students to provide anonymous feedback, they will feel more empowered and comfortable to give authentic feedback.
• Do not collect mid-semester feedback more than once, and only collect feedback that you will be able to review, actively engage with, discuss, and implement in a timely fashion (Svendby, 2021). The goal is not to overwhelm yourself with work, nor is it to collect constant feedback from students. This should be an efficient process. Additionally, when you keep asking students for feedback, it can communicate to them that their initial feedback was not valuable or seriously considered by their instructor, and can lead to survey fatigue (Svendby, 2021).
• Discuss evaluation results with students as soon as possible – preferably during the class period after the feedback was collected (Svendby, 2021). This allows students to openly discuss the feedback with the instructor, seek clarification, highlight conflicting items of feedback, and help the instructor understand what is important to students (Svendby, 2021). Additionally, it shows students that their feedback is valuable and prioritized by their instructor (Svendby, 2021). In an online course this could take place in a discussion forum, or the instructor can reply to student feedback in a video.
• Reflect critically on feedback (Svendby, 2021). You do not need to make changes simply because students have recommended you do (Svendby, 2021). Take time to think about what you can do, what is useful, and potential biases in student responses – especially regarding the instructor’s demographic background, as white men are more favored in student feedback than women and non-white instructors (Svendby, 2021). Think about what reasonable changes you can make to your teaching and course delivery that will actually benefit the student’s learning experience in your class (Svendby, 2021).

Self-Evaluations

To be an instructor is to be in a constant state of reflection and revision of one’s own teaching practices as they align with their teaching philosophies and values. Student feedback is a great way to get an external view of one's own teaching, but it is not the only, nor necessarily the most impactful, way to improve teaching practices. Self-reflection is taking a step back and examining one's own actions and motivations, how effective they are, how much they align with one’s values, and what changes the instructor can make going forward to enhance their practice (Cole et al., 2022).

Critical self-reflection is a continuous and habitual process that remains ongoing through the instructors’ careers. In addition to reflecting on course improvement, it can also enhance instructors’ cultural inclusivity, confidence in course facilitation, and reduce feelings of burnout (Taylor et al., 2021; Slade et al., 2019; Butville et al., 2021; Cole et al., 2022).

This practice can potentially be triggered by interactions that challenge the instructor or involve some form of external or internal conflict that prompt instructors to review and refine their own assumptions and practices (Russel, 2018; Cole et al., 2022). Self-reflection can be a habitual and automatic practice, or it can be self-structured. Below are some ways to aid in the reflective process of self-evaluation.

Approaches to Self-Reflection

• Keep a list of questions that can prompt the self-reflection process (Brownhill, 2022). For example, ask yourself if your lesson was effective, and list reasons why it may or may not be. If your students struggled with a lesson or assignment, ask yourself if the instructions and expectations you communicated were done so in a way that was clear enough for students. What did you excel at in a lesson or course, and what areas did you struggle with?
• Keep a journal reflecting on your teaching in a given time period (Yale, n.d.). This will ensure continuous engagement with one’s reflective teaching practices and allow a history of teaching practices to be examined and considered at a later date (Yale, n.d.). The journal entries do not need to be in-depth or laborious, but simply contain general reflections on one’s teaching and what did, and did not, go well in class that day/week/month (Yale, n.d.).
• Review video-recorded lectures, if possible (Yale, n.d.). Instructors taking time to observe their own teaching can provide an opportunity to examine what did and did not work, what they excelled at, and what they need to improve in their class delivery (Yale, n.d.).

There are several tools for instructors to choose from when approaching continuous improvement strategies. Standardized end-of-semester evaluations, mid-semester evaluations, and continuous self-reflective evaluations each possess their own strengths and weaknesses as tools but can be used individually or collectively to enhance an instructor’s teaching practice. Instructors can utilize multiple methods of teaching evaluation to continuously improve their teaching practice.

References

Boring, A., Ottoboni, K., & Stark P. B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. Computer Science. http://doi.org/10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1 

Boston University Center for Teaching & Learning. (n.d.). Getting Feedback from Students. Boston University. https://www.bu.edu/ctl/teaching-resources/start-stop-continue/

Brownhill, S. (2022). Asking Key Questions of Self-Reflection. Reflective Practice, 23(1), 57-67. doi: https://doi-org.libproxy.library.unt.edu/10.1080/14623943.2021.1976628

Butville, B., Hanrhan, S., and Wolkenhauer, R. (2021). Prepared to Take
 Responsibility: Practitioner Inquiry for Social Justice in a Professional Development School Partnership. School-University Partnerships, 14(3), 167-190.

Cole, C., Hinchcliff, E., & Carling, R. (2022). Reflection as Teachers: Our Critical Developments. Frontier Teacher Education, 7, 1-5. doi: https://doi.org/10.3389/feduc.2022.1037280 

Cunningham, C. M. & White, T. L. (2022). What are They Trying to Tell Me? Large-Scale Viability of the Start, Stop, Continue Teaching Evaluation Method. Innovations in Education and Teaching International, 59(1), 60-69. doi: https://doi.org/10.1080/14703297.2020.1810099

Diaz, N. P., Walker, J. P., Rocconi, L. M., Morrow, J. A., Skolits, G. J., Osborne, J. D., & Parlier, T. R. (2022). Faculty Use of End-of-Course Evaluations. International Journal of Teaching and Learning in Higher Education, 23(3), 285-297.

Marx, R. (2019). Soliciting and Utilizing Mid-Semester Feedback. Vanderbilt University Center for Teaching. https://cft.vanderbilt.edu/soliciting-and-utilizing-mid-semester-feedback/

Russell, T. (2018). A Teacher Educator’s Lessons Learned from Reflective Practice.
 European Journal of Teacher Education, 41(1), 4–14. doi: https://doi.org/10.1080/02619768.2017.1395852

Slade, M. L., Burnham, T. J., Catalana, S. M., and Waters, T. (2019). The Impact
 of Reflective Practice on Teacher Candidates’ Learning. International Journal for the Scholarship of Teaching and Learning, 13(2), 1–8. doi: https://doi.org/10.20429/ijsotl.2019.130215

Svendby, R. (2021). Learning by Doing it Wrong: An Autoethnography Inviting Critical Reflection of Lecturers’ Disability Awareness. Teaching in Higher Education, 26(4), 636-643. doi: https://doi.org/10.1080/13562517.2021.1872528

Taylor, L. P., Newberry, M., and Clark, S. K. (2021). Patterns
 and Progression of Emotional Experiences and Regulation in the
 Classroom. Teaching and Teacher Education, 93. doi: https://doi.org/10.1016/j.tate.2020.103081

The University of Washington. (n.d.) Developing a Reflective Teaching Practice. The University of Washington. https://teaching.washington.edu/topics/inclusive-teaching/inclusive-teaching-strategies/developing-a-reflective-teaching-practice/

Uttl, B., White, C. A., & Gonzalez, D. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning
 are not related. Studies in Educational Evaluation, 54, 22-42. doi: https://doi.org/10.1016/j.stueduc.2016.08.007

Watermark Insights. (2022). The Benefits of Course Evaluation in Higher Education. Watermark Insights. https://www.watermarkinsights.com/resources/blog/the-benefits-of-course-evaluation-in-higher-education 

Yale Poorvu Center for Teaching and Learning. (n.d.). Reflective Teaching. Yale. https://poorvucenter.yale.edu/ReflectiveTeaching