If you are reading this, you have probably figured it out. The topics discussed and covered by Bloom’s Taxonomy are all related. It is not a pyramid, it is 3D and there is some alignment between Bloom’s cognitive complexity taxonomy and the ‘new’ model of cognitive processes. If evaluating is not an explicit learning outcome, taught and studied as a separate skill, the student is not likely to be aware that there are different levels of expertise and cognitive development in play. The actual cognitive level will likely be at the novice level, at the low end of the novice level.

This note is about the specific and conscious inclusion of evaluating as a learning outcome, and recognizing that evaluating is more complex than the comprehending, applying, analysing, and synthesizing if evaluation is actually taken seriously and viewed as a cognitive skill. It is a separate skill set and there are different evaluation elements at each stage of the problem-solving journey; evaluating the level of comprehension, evaluating how to apply and evaluating the application, evaluating the process of analysis and the results of analysis, evaluating the process of creation and synthesizing, and the evaluation of the creation output. Most people do not appear to think too much about evaluating. We have.

You need knowledge and skill about comprehension to do better, more effective and efficient comprehension. You need knowledge and skill about how to apply ideas and you need to have comprehension to do this. And so on and so forth. Within each level of the taxonomy, all of the other levels of the taxonomy co-exist because each layer is a skill and a domain. Stuff like this can make the head hurt. It is like the nested dolls; within each doll there is another complete doll.

At each level in the taxonomy, you are evaluating and judging how well you are doing (or you should). There are decisions and choices to make throughout the process. The type of evaluation varies with the cognitive level and process step. Whenever you have two or more choices, and you have to choose one, you are evaluating - consciously or subconsciously. A number of ideas for how to evaluate and think about evaluating can be found in various notes and they apply as much to the students’ ability to evaluate as they do to an instructor’s.

This note is the about the highest level of evaluation complexity. This is where you have to look at the bigger picture, have the context, experience, and ability to understand what other choices existed, or could exist, and how the current situation fits into the world order. Knowing how to evaluate at this level is not learned by osmosis or just by doing. Evaluating, at the highest level, involves many cognitive skills and abilities coming together.

When it comes to teaching and assessing evaluation, it is best to position it throughout the course(s) and ask students to justify and examine their work, and then explicitly teach and explore the wild world of evaluation AFTER synthesis, when the students have actually created something to evaluate in a holistic fashion. When there might be multiple, alternative solutions or answers that can be compared for completeness, assumptions, strengths and weaknesses. This is what makes evaluating more complex. If the final level of evaluating is not taught and explored explicitly, it is not likely to become better than that demonstrated by a novice.

Learning thoughts…

• At each stage of the journey, the students should be taught to pause, evaluate, reflect, and adjust. This is also part of learning how to improve (e.g., see the plan-do-check-act cycle in the continuous improvement note ).
• Evaluation should always have a goal (why evaluate, what information is being sought through the evaluation). The goal should dictate methods, effort, evaluation skill requirements, etc.
• The final stage of evaluation should be anchored on the original issues, requirements, and objectives and it is important for students to learn that certain things should be documented at the start (or near start) as they are important and will be used later. For example, baseline measurements might be needed at the start of an activity if gains etc., are to be understood later.
• The higher the impact and potential risk, the more skill you need. Students should understand the cognitive skill level requirements, their own skills, and what are reasonable evaluations to undertake.
• Evaluations should have a baseline or benchmark if any perceptions, opinions, or comparisons are being used. Students should be taught how to source or develop comparatives.

• Evaluations need to be designed (another nested doll) and evaluated.
• In major studies, the evaluators and evaluation methods (and results) are often evaluated by domain and subject experts (yet another nesting). Students need to know that there are best practices and ways to evaluate and many mistakes to avoid.
• Evaluating carries obligations and responsibilities; to do the appropriate data gathering, analysis, and interpretation. Students should be taught each of these steps and they should practice them.
• Students should be taught how to justify and explain the evaluation process.
• Student need to be taught how to evaluate the result of a synthesis activity, how to evaluate a preliminary test, use or pilot that uses the synthesis output, initial formal use, the learning curves, and sustained use.
• Part of an evaluation might be the generation of observations in addition to a quantitative analysis, and there might also be possible recommendations or suggestions. Students should be taught how to create observations and recommendations, and how to communicate them.
• Balance is needed when doing an evaluation and it is important to keep the goal and criteria in mind. Interpretation and compromise is often necessary and students need to know when, why, and how to do this.
• It is possible that an evaluation involves stakeholders. The students need to know the various ways (assumptions, strengths and weaknesses) that stakeholders can be engaged in an evaluation (e.g., interviews, structured interviews, surveys, focus groups).
• Not every stakeholder is equal in terms of quality input (e.g., knowledge, bias, attitude, perceptions vs reality) and students need to learn how to listen to everyone, and believe no one until there is evidence of some kind. They need to know about the potential for bias, how to recognize it, and how to deal with it.
• It is unlikely in practice that sufficient time and resources will exist for ‘best practice’ evaluations. It is also common that an evaluation needs to be speeded up with a changed deadline or goal. Students need to know how to react to such situations.
• There are different styles of summative evaluations, but when possible, they should focus on empirical or quantitative elements and yield insights which are actionable. The evaulating should be conscious and aware of possible biases such as introduced by opinions, influence from others, and possible errors that can arise when picking comparators, baselines, and evidence.

Assessing thoughts…

• Many of the same, general points from 8.1-8.5 apply. To make them stick and help you understand, now is the time to re-read those pages.
• You can ask the students to develop rubrics and evaluation criteria and justify their answers.

• You can provide rubrics and ask the students to evaluate them.
• The students can be asked to evaluate one or more 'solutions' and break it down into the criteria used, why was the criteria used, etc.
• You can ask the students to develop evaluation criteria for evaluating a evaluation criteria.
• You can have them evaluate an evaluation criteria (perhaps generated by another student or student team) and ask them to compare their evaluation criteria to that provided. This is the meta-cognition stuff legends are made of.
• You need to debrief and discuss the evaluation process and take class time to do it. Especially after an assessment. And, then alllow the students another attempt to demonstrate their understanding about evaluation.

Further reading

• Anderson, L., & Krathwohl, D. A. (2001). Taxonomy for learning, teaching and assessing: A revision of Bloom's taxonomy of educational objectives. Addison Wesley Longman.
• Maranan, V.M. (2017). Basic process skills and attitude toward science: inputs to an enhanced students' cognitive performance, Thesis, Master of Arts in Education, Laguna State Polytechnic.