Ok, you have been asked to do a peer review or to help a peer. The peer might have approached you out of a sincere, continuous improvement mindset or because of recent feedback from the students or the chair. The chair might have asked you to help the peer because of some perceived issue. The review might also be part of a formal review for merit.
You teach, you have been teaching for many years, obviously you know how to evaluate and how to discuss the evaluation with your peer. Or do you? Unfortunately, many instructors have never been actually trained on how to do a process evaluation, how to evaluate something like a course, what to evaluate, how to analyse the evaluation data, or how to convey the results. There is an added complication; there is supposed to be something called Academic Freedom for research, but this is often applied to teaching as well, not just the what, but the how as well. This does not make 100% sense, but it seems to exist in many situations we have had exposure to. Not done yet, there is also the collegial thing, member of the same tribe, coffee buddies, and such. These can also make it hard to help a peer. How much can you say, what should you say, how should you say it, when should you say it?
In theory, the instructor is 'expert enough' in the domain to teach the course, answer questions, and explain the subtleties of the topic. In theory, the instructor is working off of an approved course description and other department/program documents that describe the learning outcomes and institutional expectations of what is taught and what is learned. Usually the 'how' is left up to the instructor, but the official paperwork might also specify things like labs, etc. In theory, the instructor knows the appropriate pedagogical methods and techniques that match the course outcomes, material, and incoming student characteristics. In theory, the instructor knows how to use the methods, when to use them, when not to use them, and can execute the methods in a live fashion, at a level of competence expected by the institution. In theory, the instructor knows how to manage the classroom, facilitate the learning, and design, execute, and interpret the appropriate assessments. In theory, the instructor knows how to write a course outline and plan a course. That is a lot of 'in theory's; six, and they are compound.
Three things about the above paragraphs. At least three. But we will start with three biggies. First, how does an instructor actually learn and develop all of the skills implied by the 'in theory's? Second, in professional evaluations, it is assumed that the person evaluating has all of the 'in theory' bits covered and at a higher level than then the person being evaluated. Yep, all and at a higher level, and know what would be considered best practice and industry common practice. Deeper and broader. Where and how is this learned? Developed? Assessed? Third, how does the reviewer learn how to review and evaluate? Fourth, the reviewer also needs to know about the domain and the program (fine point).
Do not get us wrong. We believe in peer reviews, but we also understand the challenges and issues. It is hard. It is hard for an instructor to do a good job because the system does not usually exist to develop the skills. It is hard for the reviewer. We have been involved in many peer reviews over the years. We have been quite lucky to have great peers and fantastic peer review experiences.
There are some things that can go wrong or make the peer review problematic. The one being reviewed might feel threatened, be defensive. If not related to promotion or retention, this situation suggests a very gentle leading to a better solution (if required) and not a full-on critique or evaluation.
It is also possible that someone else, not necessarily an 'expert' or schooled in evaluations has provided feedback already and gave a thumb's up. Everyone is an expert it seems when it comes to teaching and how it should be done, and if two birds of a feather are looking at the same thing, you will get confirmation bias and a I love you, you love me situation. If the helpful friend does not really have a clue, they might be impressed. If it is similar to what they do, they will also be happy. This can create a problem when someone else comes along to do a review. They might be wondering why the chair is suggesting help; everything was fine, the buddy said so.
New faculty might also have been told by someone what to do, and they were following an existing course or instructor's methods without knowing if it was good stuff or not. This will also make it awkward if someone has linked up with someone who should not be giving advice. But, in the world of academia, everyone is willing to give advice, even if they are not self-aware. For new faculty, we suggest that they listen to all, trust none until suggestions are bounced off someone from the institution's centre for teaching support or the department's teaching expert. Of course, caution is needed because the local teaching champion's reputation might not be based on actual knowledge and skill at a 'best practice' level, but that is another story.