Ok, you have been asked to do a peer review or to help a peer. The peer might have approached you out of a sincere, continuous improvement mindset or because of recent feedback from the students or the chair. The chair might have asked you to help the peer because of some perceived issue. The review might also be part of a formal review for merit.

You teach, you have been teaching for many years, obviously you know how to evaluate and how to discuss the evaluation with your peer. Or do you? Unfortunately, many instructors have never been actually trained on how to do a process evaluation, how to evaluate something like a course, what to evaluate, how to analyse the evaluation data, or how to convey the results. There is an added complication; there is supposed to be something called Academic Freedom for research, but this is often applied to teaching as well, not just the what, but the how as well. This does not make 100% sense, but it seems to exist in many situations we have had exposure to. Not done yet, there is also the collegial thing, member of the same tribe, coffee buddies, and such. These can also make it hard to help a peer. How much can you say, what should you say, how should you say it, when should you say it?

In theory, the instructor is 'expert enough' in the domain to teach the course, answer questions, and explain the subtleties of the topic. In theory, the instructor is working off of an approved course description and other department/program documents that describe the learning outcomes and institutional expectations of what is taught and what is learned. Usually the 'how' is left up to the instructor, but the official paperwork might also specify things like labs, etc. In theory, the instructor knows the appropriate pedagogical methods and techniques that match the course outcomes, material, and incoming student characteristics. In theory, the instructor knows how to use the methods, when to use them, when not to use them, and can execute the methods in a live fashion, at a level of competence expected by the institution. In theory, the instructor knows how to manage the classroom, facilitate the learning, and design, execute, and interpret the appropriate assessments. In theory, the instructor knows how to write a course outline and plan a course. That is a lot of 'in theory's; six, and they are compound.

Three things about the above paragraphs. At least three. But we will start with three biggies. First, how does an instructor actually learn and develop all of the skills implied by the 'in theory's? Second, in professional evaluations, it is assumed that the person evaluating has all of the 'in theory' bits covered and at a higher level than then the person being evaluated. Yep, all and at a higher level, and know what would be considered best practice and industry common practice. Deeper and broader. Where and how is this learned? Developed? Assessed? Third, how does the reviewer learn how to review and evaluate? Fourth, the reviewer also needs to know about the domain and the program (fine point).

Do not get us wrong. We believe in peer reviews, but we also understand the challenges and issues. It is hard. It is hard for an instructor to do a good job because the system does not usually exist to develop the skills. It is hard for the reviewer. We have been involved in many peer reviews over the years. We have been quite lucky to have great peers and fantastic peer review experiences.

There are some things that can go wrong or make the peer review problematic. The one being reviewed might feel threatened, be defensive. If not related to promotion or retention, this situation suggests a very gentle leading to a better solution (if required) and not a full-on critique or evaluation.

It is also possible that someone else, not necessarily an 'expert' or schooled in evaluations has provided feedback already and gave a thumb's up. Everyone is an expert it seems when it comes to teaching and how it should be done, and if two birds of a feather are looking at the same thing, you will get confirmation bias and a I love you, you love me situation. If the helpful friend does not really have a clue, they might be impressed. If it is similar to what they do, they will also be happy. This can create a problem when someone else comes along to do a review. They might be wondering why the chair is suggesting help; everything was fine, the buddy said so.

New faculty might also have been told by someone what to do, and they were following an existing course or instructor's methods without knowing if it was good stuff or not. This will also make it awkward if someone has linked up with someone who should not be giving advice. But, in the world of academia, everyone is willing to give advice, even if they are not self-aware. For new faculty, we suggest that they listen to all, trust none until suggestions are bounced off someone from the institution's centre for teaching support or the department's teaching expert. Of course, caution is needed because the local teaching champion's reputation might not be based on actual knowledge and skill at a 'best practice' level, but that is another story.

Suggestions for an informal, collegial review...

• We like to start off with asking why they are asking for help, what has triggered the request. We follow this up with asking them to describe the course to us, communicate the learning outcomes, and any assumptions.
• It is good to know where they started, what they inherited, why the course is what it is. What has been tried, what has not been tried. Do some intel before making any assumptions, reaching conclusions, or making suggestions!
• Work your way through the material, asking the instructor to explain, talk it through, why decisions were made, why certain methods are used. Do not be judgemental or make comments at this point, just keep talking, asking, reflecting.
• Try to ask 7-10 'dumb' questions leading the instructor to self-realization and the moment of enlightenment. It is better if the instructor comes to the realization themselves that things change, should be changed. Remember, it is about them, not you, and should not be used as an opportunity to show how smart you are.
• Offer to continue to help the instructor think it through, the implementation, offer to observe, review material during the transition, the post-review of the change.

• In some cases, ask the instructor about some of your own course decisions, break the ice so that it is not just about them, but it is something each of you is doing to help the other.

Suggestions for a more formal, structured review...

• Start with the course learning outcomes; what is the course supposed to achieve. This will be your rock.
• Next, review the course plan and overall schema in light of the learning outcomes.
• Third, from a teaching dossier assess the alignment of the pedagogy with the outcomes, material, and incoming student assumptions. If a teaching dossier does not exist, a long discussion will be required.
• Be fair and use good comparators and baselines; do not compare apples and oranges. Each course is different and it is important not to project pet ideas or have a fixed-in-concrete, one-size-fits-all view of how a course should be taught.
• The assessments need to be reviewed for alignment with the learning outcomes and suitable pedagogical foundation; that is, do the assessments make sense, are fair, and actually provide an assessment of the students' learning from the course.
• Perform multiple in-class observations; start of term, mid-way, end of term.
• Write up an evaluation focusing on the learning outcomes and the course's approach to delivering them and the execution. Discuss the pros and cons in each, noting any benchmarks or comparators being used. Make recommendations and if necessary, provide a strategy and plan for moving ahead.

Further reading

• Susan Thomas, S., Chie, Q.T., Abraham, M., Raj, S.J., Beh, L-S. (2014). A Qualitative Review of Literature on Peer Review of Teaching in Higher Education: An Application of the SWOT Framework. Review of Educational Research, 84, 1, pp. 112-159.
• Cosser, M. (1998). Towards the Design of a System of Peer Review of Teaching for the Advancement of the Individual within the University. Higher Education, 35, 2, pp. 143-162.
• Bianca Fileborn, B., Wood, M., and C. Loughnan (2022).Peer reviews of teaching as appreciative inquiry: learning from “the best” of our colleagues. Higher Education, 83, pp. 103–117.
• Grainger, P., Crimmins, G., Burton, K., F. Oprescu (2016) Peer review of teaching (PRoT) in higher education – a practitioner’s reflection, Reflective Practice, 17, 5, pp. 523-534.