Reappraising ‘critical analysis’ and how we make use of it:
In 1956 Benjamin Bloom edited the first volume of Taxonomy of Educational Objectives: The Classification of Educational Goals, which outlined a classification of learning objectives that has come to be known as Bloom’s taxonomy. Most assessment objectives in education are now derived from Bloom’s taxonomy, and usually from the categories in its cognitive domain.
Bloom’s taxonomy classifies verbs based on the level of difficulty of the thinking skill required by the activity in question. ‘IDEAL’ is a handy acronym for remembering the verbs in order of least to most demanding; Identify, Describe, Explain, Analyse, Link. The taxonomy is used to plan and execute classroom assessments and questions, to structure feedback, to define learning outcomes, to deliver challenges appropriate to each learner’s ability, and to set suitable differentiated improvement targets so that students can progress in ability to higher categories of task and achievement (Chandler-Grevatt 2018: p.61-66).
Fig. 1: A common representation of verbs distinguished by level of difficulty on Bloom’s cognitive taxonomy.
The cognitive domain works particularly well in assessing students in sciences, maths and humanities. In other subjects such as art, drama or music, physical or creative subjects, Bloom’s psychomotor or affective domains may be more appropriate foundations for planning. However, it is likely that any written work assessed in these subjects will be done so using the cognitive categories.
While it is commonly understood that it is the teacher’s job to help students ascend this hierarchy of cognitive abilities, actually getting them to do so can be an arduous task made all the more difficult by the fact that our students come to us from a wide range of backgrounds and educational institutions with diverse experiences and abilities. From the first term a few students may be able to engage in higher order thinking and writing and it is clear that all we have to do is keep giving them the material to work with and, all things being equal, they will do the rest. For others, helping them achieve the foundations of knowledge and understanding from which they can access new subjects requires creative teaching, active learning and carefully thought-through assessment and guidance. Even then there are some, hopefully in the minority, who still struggle with the higher-order activities by the time they reach their final summative assessment.
Apart from the setting of objectives and the structuring of learning, formative writing assessments have tended to focus on using writing frameworks to help bridge the gaps between the least and most able. Typically frameworks help students to understand how to structure paragraphs and how paragraphs function together in an essay, ideally one that can be reproduced under the time constraints of the exam. Some variation of the ‘PEEEL’ acronym is most commonly used to help students remember what is required of them; Point; Explain; Example/Evidence; Evaluate; Link. As with the ‘IDEAL’ acronym mentioned above, the goal is to cover and display the range of abilities identified in Bloom’s hierarchy.
However, when assessing the fruits of these writing activities it is often hard to shake the suspicion that some students have reproduced the appearance of being able to think/write rather than genuinely displaying that ability. Teachers are often left wondering if these students could respond to unexpected questions and therefore try, often at great cost to their own health and wellbeing, to assess them on every conceivable question. Even after all of that effort, students walk away from exam halls reporting back on questions that were not previously considered, leaving their teachers second-guessing their own planning and hoping that the paper challenged students of all centres equally. On pulling the papers of students who have underperformed it is common to read answers that fail to address the question or just have too much knowledge and not enough evaluation. Then comes the bitter self-reflection and recrimination; ‘I taught them that, they could have answered it’, or ‘It is on the specification but I didn’t set it as an assessed question. I’ve let them down’.
It is no doubt true that writing frameworks can be very helpful in planning formative assessments. They help disorganised students to organise their thoughts and writing, to reflect on the different types of activity that they engage in when writing, to assess their own ability to undertake these tasks, to set themselves targets for improvement, and to try again. However, having struggled with this problem for well over a decade, what has started to become clear is that they do not really help students understand how to move from the foundational knowledge at the bottom of the skills hierarchy to evaluative judgements at the top. While classroom planning is structured around activities that progress up the ladder of difficulty, writing frameworks encourage our students to think about writing as if each cognitive activity is distinct from the others. The writing often appears formulaic rather than a genuine attempt to grapple with the problem in the question.
As a result, much of the published guidance on how to achieve success in exams focuses on understanding what the examiner wants and learning how to satisfy that developing the skills on which the student will be assessed. In the most excessive cases of the pre-linear specifications this extended to question spotting, encouraging students to focus on those areas of the specification that had not been examined recently. Guidance on exam success might include tables of past paper questions, encouraging students to prepare primarily for the types of questions that had not been asked recently (Baron, P. 2013: 67). Teachers considered it a part of their job to play the exam, prepping their students by getting them to write and learn only those answers that were needed to guarantee the chance of an A grade, and routinely ignoring the most challenging units of the course if students could focus on specific questions and do well. Where departments were successful there was often little that was transferable in terms of practitioner skill between colleagues or that students could carry away to the next stage of their education, the goal being very specifically to deliver a grade, and a grade in that subject.
Some advocates of Problem-Based Learning (PBL) go further, arguing that such practices are symptomatic of an education system that could not produce genuine student development and is fundamentally structurally unsound. David Leat argues that ‘teaching to the test’, schools ‘learning to game the system’, which in extreme cases can ‘descend into cheating and dishonesty’, is symptomatic of an educational system that requires schools to compete for parents and their children on the back of exam results. To borrow a phrase coined by Bob Lingard (2007), they are practicing ‘pedagogies of indifference’, in which every class must achieve predetermined learning objectives but there is no connection between episodes of learning, few connections to daily experience, personal interest, or contemporary issues, teaching to students who rarely ask meaningful questions, who receive knowledge over which they have no control or choice, and ‘nothing of substance is produced except work for the teacher to assess’ (Leat, 2017; p.6).
Leat argues that while ‘those with better social and cultural capital can take the best from the system and compensate for its negative effects’ (ibid; p.18), it actively disadvantages those without. Disadvantaged students are more likely to be disengaged, more likely to engage in truancy, leave school without employment or further education or training, and are placed at greater risk of poverty, and of passing those disadvantages on to further generations (ibid; p.10). For those students who feel a lack of connection to school they are more likely to experience mental health problems and suffer more emotional distress, have peer relationship problems, low self-esteem, low expectations for the future and engage in substance abuse (ibid; p.13). Without the social capital to aspire to rewarding routes of employment, students often have unrealistic career aspirations in media, sport or law (ibid: p.16).
However, from the perspective of those who advocate PBL an education system that does not challenge students to develop their understanding through the use of their knowledge can also fail those students that appear to excel within it and the professions into which they progress. In the 1960’s, McMaster University School of Medicine in Canada found that their medical students were unable to translate the vast volumes of abstract theory that they had learnt into practice knowledge and applicable skills in clinical settings. They found that ‘large volumes of pre-clinical teaching inhibited students’ sense-making capabilities and blurred their clinical judgements… [An] incongruence emerged between what was taught and what was eventually required by the curriculum and professional bodies’;’vast amounts of abstract theory did not produce the skilled and fit for practice medical practitioner required at the completion of the course’ (Morales-Mann & Kaitel, 2001, and Price, 2003, cited in Mantzoukas, 2007; p.245-6). The fact that governments of a variety of persuasions have for at least the last 20 years pressurised educationalists to translate learning into employment skills shows that for the most part little has been perceived to have changed.
I contend that this shortfall in our practice is because as teachers we have not thoroughly understood the nature and demands of critical analysis, and because of this we have not fully appreciated the relationships between the different levels of competence represented on Bloom’s taxonomy. Consequently, it seems that we have not fully understood what is required of students at key levels of difficulty. This lack of understanding has hampered our ability to consciously move students from knowledge, understanding and application, to the higher levels of analysis and evaluation.
This may be because we have been encouraged to think of each assessment objective as defining their own (largely isolated) level of competence and type of classroom or written activity, rather than steps in the building exercise they should be whereby each new ability opens up further levels of skill. As Coe appears to state above, I believe that this is because we have lacked an overarching pedagogy by which we can make sense of the functions and interrelation of such activities in the learning process; we have no framework to explain why we’re doing what we’re doing and when we should be doing it. However, by developing this understanding teachers can open up new dimensions to their teaching and assessment, take greater control over their classrooms and achieve the cathartic discovery that when they have facilitated this development, students of a range of ability are easily able to access these higher levels of skill through practice.
In the specification of one of the subjects that I teach, analysis forms its own assessment objective representing one third of the marks available. However, when I ask students, ‘what does analysis mean?’ or ‘how do you analyse something?’, quite often the question will meet with a blank response. When they are forthcoming, replies usually centre on the idea that it involves ‘detailed examination’. Whether they are conscious of it or not this derives from the etymology of the term from the Greek ‘ana’ (up) and ‘luein’ (loosen), ‘unloosen’ or ‘the resolution of anything complex into simple elements’, the opposite of synthesis (Harper 2019). This is what we mean by detailed analysis, and associated verbs might include ‘inspect’, ‘examine’, or ‘explore’.
Fig. 2: A detailed, descriptive, deconstructive, primary or first-order analysis of an engine.
This is probably what we think of first and foremost when we think of analysis. When we intentionally develop the ability in our students it might be in order to analyse prose or poetry in English, or some source material in History or Politics. Alternatively we may introduce a topic by throwing students a problem, fact or case study and then ask them to (first-order) analyse it. However this is usually to stimulate interest and will probably be followed at some point by our own analysis and that of relevant academics.
Nevertheless, when faced with exam questions that challenge students to engage in evaluative tasks, detailed analysis appears on the surface unnecessary. Being able to identify, define, and describe the salient features of an object upon an initial examination is not necessarily going to open up a judgement about its effectiveness. To do so requires that students bring additional knowledge and skill to the process, identifying relationships between what they are being presented with for the first time and their prior experiences or the material that they have previously studied and evaluated in their learning. In order to reach an evaluative judgement we need to engage in critical analysis.
Before investigating the process and functions of critical analysis, it is worth reviewing current understanding and resources. Dictionary accounts of ‘critical analysis’ refer to analysis of merits and faults, or critical thinking whereby we achieve objective analysis and evaluation. Critical analysis and its role in problem solving is highly valued by employers who have come to test its capacity in applicants by throwing them analytical challenges in pre-employment tests. However, when we look into their descriptions of the skill we find very general, ambiguous definitions followed by specific examples of its use in industry (Target Jobs, 2019). This is mirrored in educational resources where we often find verbs describing acts of detailed analysis mixed indiscriminately with those used in critical analysis, followed with examples of what a student might be doing when engaged in that act in their studies (Blake, 2017). At no point have either employers or educators explained to students the processes involved in critical analysis in the same way as they have with, for example, those involved in learning factual information. I believe that real-life examples can be useful, but only in as much as they reveal the underlying processes involved in critical analysis and how we can engage in these activities in general.
A definition of critical analysis that appears to get closer to what we seem to mean by the term might be, ‘an analysis that culminates in a greater critical appreciation of the object under investigation, where ‘critical appreciation’ stands for capacity for evaluative judgement.’ In her study guide, ‘Critical Thinking Skills; Developing Effective Analysis and Argument’, Stella Cottrell writes that in the context of academia, ‘’criticism’ refers to an analysis of positive features as well as negative ones. It is important to identify strengths and satisfactory aspects rather than just weaknesses, to evaluate what works as well as what does not. Good critical analysis accounts for why something is good or poor, why it works or fails. It is not enough merely to list good and bad points’ (Cottrell, 2005: p.8).
Cottrell distinguishes critical analysis from descriptions which ‘give an account of how something is done, or what something is like’ and ‘do not give reasoned accounts of how or why something occurred nor do they evaluate outcomes. In reports and academic writing, description should be factual, accurate and free of value judgements.’
‘Description is sometimes confused with critical analysis as both can investigate an issue in detail. Descriptive detail is not intended to persuade to a point of view but aims, rather, to give the audience a more thorough impression of the item or issue being described.
Example: The solution was placed in a test-tube and heated to 35 centigrade. Small amounts of yellow vapour were emitted. These were odourless. Forty millilitres of water were added to the solution, which was then heated until it began to boil. This time, grey steam was emitted. Water droplets gathered on the side of the test-tube.
No reasons are given for what happened. That critical analysis of the results would be in a separate part of the report.’ (Cottrell, 2005; 54).
Cottrell does not offer her own definition of ‘critical analysis’ preferring instead to define it by means of activities which are captured by the use of the term, but it is clear that she means something approaching the definition given above. However, while this might resolve some confusion over what it is and is not, it appears to further blur the lines between detailed analysis and quite distinct critical activities. There is a line that exists between the two and a point of transition from one to the other. What we seem to lack is a clear appreciation of how we help students traverse this gap.
We can start to understand the nature and process of critical analysis by distinguishing it from detailed description and evaluation. A simple activity that I use to help students to understand the differences and relations between descriptive, analytical and evaluative writing illustrates the point. I ask students to examine the following image:
Fig. 3: Emmet’s double-sofa from the LEGO Movie.
I use this image because it is simple thus making it easy to describe, has noteworthy features making it easy to analyse, and is a novel solution to a problem, how to sit lots of people in the same space, thereby inviting evaluation. I give students 2 minutes to describe the object in the image. Their answers usually observe the following features; ‘It’s a sofa made of Lego. It’s one sofa on top of another one. It has ladders up the side to access the sofa above. It has cup holders on the arms of the sofas. It is blue, grey, and brown, and the cups are white’. I then ask them to analyse the sofa by noting anything of significance about it. Responses tend to include, ‘The legs of the people sitting on the sofa above would dangle in the faces of those below. Their drinks might fall on them too. It doesn’t look very stable and could topple over.’ Finally I ask them to judge its effectiveness as a sofa. Obviously students identify that it wouldn’t perform well as a sofa (beyond the world of Lego) because it is uncomfortable and potentially dangerous. They typically note that these flaws outweigh the single advantage of space saving, and sometimes go on to point out that two sofas would be better than this one.
Firstly, when we clearly distinguish descriptions, analysis and evaluations, students can easily see the quantitative and qualitative differences between these styles of thinking and writing. This is a useful first-step toward moving beyond descriptive writing as it shows that analytical writing is of a different order, being much more focused on particular aspects of the object that warrant our attention. Secondly, the example also shows students that there is a close relationship between analysis and evaluation; because the features in the example above are made significant by their weaknesses, the subsequent evaluation is and can only be largely negative. However, it also reveals a great deal about the nature and practice of critical analysis as distinct from detailed analysis. Unlike the latter which is largely investigative, the student is bringing something more of their own understanding of the world to this analysis, in this case their ability to imagine it in a real-life situation and how it might function in its role. Critical analysis appears to involve students using their knowledge in a critical dialogue between their understanding of the world and the object under analysis.
Colleagues that have begun to adopt the theory that I am proposing here have developed the activity above in original ways. In a skill-development session one teacher sets questions along the lines of, ‘‘Houses made of sticks provide an adequate means to protect against wolf attacks.’ Assess this claim’. It helps students to clearly distinguish between knowledge, critical analysis and evaluation, and shows them that from the perspective of someone with a sound foundation of knowledge many of the evaluative questions set by examiners contain propositions that appear similarly naive, and therefore why a strong background of knowledge is necessary to see this and respond to it. More importantly however, it identifies the problem in the question, in this case how to protect oneself against a highly motivated wolf.
The central thesis of this essay is that what distinguishes the activity of critical analysis from other forms of activity like detailed analysis or critical thinking is that it involves problem solving. When asked to ‘critically analyse’, the student is being challenged to reach an evaluation. When asked, ‘is this good?’, the problem students are being presented with is to determine whether that thing is or is not in fact good, and they do so through the skill of being able to identify what pieces of knowledge from those that they have accumulated are relevant to the problem at hand. Therefore, critical analysis is motivated by a challenge or problem in relation to which it serves a diagnostic function.
Imagine you have a jar of your favourite jam. It’s brand new, unopened, and as you start to twist the lid you are imagining peeling back the greaseproof paper from the glistening preserve, scooping out knife fulls of jelly and spreading them thick on hot buttered toast. As your mouth begins to water you realise that the lid won’t budge. It’s as if it’s glued on. There’s no way even that would stop you now, so you scour the recesses of your mind for solutions. Could you wedge the lid between two surfaces instead of holding it and focus all your force on twisting the jar? Do you have a monkey wrench that is wide enough to grip the lid giving you more leverage to work with? Then you remember that when things heat up they expand and you think, if the lid expands it won’t grip the jar so tightly. After considering using the oven or boiling water you opt for the hot water tap, and within a few minutes you are enjoying your toast and jam, albeit not quite as you imagined it would be before the drama with the lid. What you have done is move from acquired knowledge to the use of that knowledge which would have languished unutilised if it had not been for a pressing problem and the motivation to solve it.
I will be asserting that when thought of this way, teachers can easily draft resources that make the most of critical analysis and can see how other activities and resources involving detailed analysis and critical thinking or reflection relate to and can be used in conjunction with critical analysis. Moreover, I am arguing that such phases in the learning cycle share all of the characteristics of problem-based learning which has until now been associated predominantly with vocational aspects of education in subjects like medicine, law and business. As it has been utilised previously in problem-based learning students have been encouraged to use their accumulated knowledge and thereby develop cognitive skills, but this has been seen as distinct from and often antithetical to knowledge acquisition. However I am asserting that such activities play an integral role in knowledge acquisition and when this is recognised, all of the benefits that are associated with problem-based learning accrue to mainstream education.
Before I became aware of PBL or the medical teaching at McMaster University I had been engaged in action research to improve the quality of student engagement with subject content and had come to use vocational analogies to help explain to students the difference between knowledge acquisition and critical analysis. To show them how they are expected to use their knowledge to address challenges set by questions, I compare them to people who are expected to find solutions to problems in the workplace. For example, a GP will go through 6 years of basic medical training before having to go through a further 6 years of specialised training. This will qualify them to diagnose conditions and identify appropriate treatments. However, when we bring our medical complaints to our GP, we don’t expect them to tell us everything that they know about such ailments; a GP is not going to list everything they know about chest pains if that’s what we are complaining of. Our doctor is going to focus an examination on the area of concern and then relate what they know to what we have presented them with, and doing so should suggest a solution. Similarly, when we complain of a knocking sound coming from our car’s engine, the motor mechanic isn’t going to tell us why engines produce knocking sounds, and if they did we would likely find it frustrating. The mechanic would use their knowledge to focus their investigation on what is causing this knocking sound in this engine.
These workplace examples involve both detailed and diagnostic analyses, and I will discuss the relationship between the two in educational settings below. Primarily what they reveal to students is how knowledge is used diagnostically in critical analysis. Secondly however they are useful in showing that the knowledge being used pivots on the problem being presented. For example, essay questions that require analysis and evaluation are asking students to use their knowledge and understanding to diagnose solutions to the problems presented in the question. Subtle changes in the problem being presented can require significant changes to the knowledge appropriate to the task. If I complain of a chest pain to my GP, the diagnosis will be significantly different depending on where exactly that pain is in my chest. This reveals to students the importance of developing mental dexterity through practice in preparation for such tasks.
Returning to the sofa activity described above, the critical emphasis appears to be on its functionality as a sofa; is it good at what it is attempting to do? When we ask students to do the same with argument or theory, we are asking students to critically emphasise those aspects that distinguish them from others. The problem again is one of functionality; is the argument good at what it is attempting to do? What something is trying to achieve can be regarded from the perspective of the problem that it is trying to address. It may be good at one thing, but less so good at another. The more specific the problem the more efficiently students need to use their knowledge to address it, only emphasising the aspects of the object that are relevant to that problem. For example, an engine might be great in terms of durability without being powerful – if you are looking for a car to last a long time, the former consideration is going to trump the latter. If the problem is only focused on how well it achieves one of these things then the other may be irrelevant to the evaluation. As the problem being addressed changes, so too does the analysis.
Analogies such as those above help students to see that there is a significant difference between knowing and understanding something, and being able to use that knowledge and understanding to overcome problems. They can also be used to explain that exam questions are like problems or challenges that require students to produce diagnoses and recommend treatments, and that small differences to exam questions can require the use of very different items of knowledge. Importantly however, this also reveals the importance of motivation, focus and attention to detail. Problems are rarely solved by people who do not see them as problems or who perceive them to be an unwelcome distraction. We’d get angry at a mechanic who didn’t listen to us because they were on their phone or dismissed what we’re saying by telling us that we were probably wrong, but we’d be apoplectic and not a little worried if this were our GP. The utility of these analogies is that they can be tailored to individual students by using their own choice of career path; I have used examples as diverse as medicine, motor-mechanics, the rag trade and coaching athletes. Even so, I have found in general it is the first of these that has the greatest rhetorical impact on students.
Critical analysis or diagnostic analysis – using analysis to solve a problem – requires a different set of activities to those involved in detailed or descriptive analysis. Important primary verbs are ‘identify’, highlight’ or ‘relate’ as they are the tasks that students engage in when relating aspects of their previous learning to the question or problem at hand, identifying the knowledge that is appropriate to the task. In these activities students identify the ‘tools’ appropriate to the problem; the problem defining the knowledge and understanding that is appropriate, with changes to the problem requiring the student to establish different relationships with different items of knowledge. Once appropriate knowledge has been identified, the student will still be required to show how these various items of knowledge ‘relate’ to each other, but other relational verbs covering the use of knowledge may be more appropriate. For example, students may have to ‘differentiate’, ‘distinguish’, ‘contrast’, or ‘compare’ the most salient points they initially identify as relevant to other aspects of their learning, engaging in such comparative analysis particularly if required to reach a balanced judgement. While these verbs can also be used to stimulate more detailed analysis, e.g. ‘highlight the moving parts in this diagram and distinguish those that rotate from those that don’t’, unlike the verbs used in detailed analysis they can also be used to initiate first responses to problems or challenges, e.g. ‘identify the set of claims regarding this problem that appear to be most accurate and contrast them with those that appear to be weaker’. Unlike a descriptive analysis which is not specifically related to prior learning, diagnostic critical analysis usually suggests a solution. This is because detailed prior knowledge gives us an understanding of what works, what does not, and where potential weaknesses might be found. We will examine some of the classroom resources that can be used to develop this skill in students below.
It should be noted here that the activities just described are related to but far simpler than those we traditionally think of as critical thinking skills. The ability to identify the relevance of some piece of information to a problem at hand is much more closely related to one’s immediate background knowledge and understanding than the skills of deductive and inductive reasoning or reflective self-analysis which involve the art of manipulating that knowledge to greater effect. Before students can utilise these skills (if they have consciously developed them at all) they must be able to identify something that could be used and say why it should be used. This I would argue is the basis and first step in the process of critical analysis and the activity is much closer to the roots of analysis in understanding. Moreover as I will argue below, the place and function of critical thinking skills within the curriculum is made more coherent when they are brought into the service of these more basic intuitive core activities involved in critical analysis.
To start the process of critical analysis students must have some solid basis of knowledge from which to work. Marking the work of a student who is trying to analyse and evaluate something that they don’t understand can be the most challenging aspect of the marking process, with the teacher trying to award marks but being held back through the student’s lack of clarity. It is as if they are trying to identify the relevant aspects of something that they only see through a fog. However, when they do have a solid foundation it is perfectly possible for them to use this knowledge critically to effectively address challenges without having to consciously adopt more refined critical techniques, and as an activity it must take place much earlier in a student’s educational development than it would if we confused critical analysis with ‘critical thinking’ as many tutors or academics are inclined to. This essay will return to consider the importance of foundational knowledge below as it has been recognised as being one of the weak-links in the delivery of PBL.
In order to contrast the interpretation of critical analysis given above with other accounts, and how I believe it can more efficiently redescribe the activities those authors define as critical analysis in the learning cycle, it is necessary to consider how redefining critical analysis as the central activity within problem-based learning cycles also reimagines the place of the learner within that process. One of the strengths of PBL is its emphasis on the personal development of the learner; rather than their acquisition of knowledge PBL focuses on how the knowledge they acquire changes the learner. Savin-Baden (2000) describes the PBL ideal as follows; ‘Learning should be seen as a cyclical process in which students make transitions through which they develop increasing (and also sometimes decreasing) understandings of themselves, their context, and the ways and situations in which they learn effectively.’ (Savin-Baden, 2000; p.9). Professor Savin-Baden associates PBL with the work of Dewey for whom knowledge is ‘not as something that is reliable and changeless but as something that is an activity, a process of finding out… [students] are not spectators, but agents of change.’ (ibid; p.4).
Savin-Baden points to similarities with socratic dialectic in the way PBL encourages students to question assumptions, values and the adequacy of solutions, dispose of contradiction and set aside appearance and received opinions; ‘This kind of increased understanding and examination of perspectives and frameworks is encouraged through problem-based learning because it offers students opportunities to examine their beliefs about knowledge in ways that lecture-based learning and narrow forms of problem-solving learning do not.’ (ibid; p.3). Leat agrees that the nature of enquiry- and problem-based learning is that they…
‘stress a personal engagement with, and construction of, the world as perceived by the learner… the ontological standpoint does not suggest that there is an entirely objective world, but one in which there is some degree of knowable reality that is being gradually discovered, but that this is understood and constructed by people in individual and social ways. This is termed a critical realist view… results of enquiry change us, change the self, in terms of how we think, feel or act. These changes can be large or small, and may at some point constitute transformation.’ (Leat, 2017; p.33).
We can see here from the definition of this ontological perspective that Leat ascribes to PBL (ontology in this case being concerned with the nature of what it is be the learner in the process of learning), that the acquisition of objective fact is regarded at best as a debatable aspiration, and that the goal of the learning process is the transformation of the learner; ‘Transformative learning (Mezirow, 2000)’ involves ‘qualitative changes in ‘meaning perspectives’, ‘frames of reference’ and ‘habits of mind’ (1978, 1991)’ and ‘is understood to involve substantial alteration to a person’s concepts, values, feelings and behaviour.’ (ibid; p.34).
Critical analysis, understood as incorporating PBL, facilitates a dialogue between knowledge and evaluation that transforms the learner in these ways. Unlike detailed analysis which at most could be described as tacitly dialectical, diagnostic analysis and its relationship to evaluation is overtly dialectical. Dialectic method (as opposed to didactic lecturing) concerns how judgements are arrived at through dialogue and debate between one or more participants, whereby the interaction between them produces a change in the other. Although the philosophy of Aristotle, Hegel, Marx or Vygotsky include much more sophisticated variations on the theme of dialectics, in order to grasp the form of the dialectical process teachers need only consider the processes they use in action research or Kolb’s experiential learning cycle.
Fig. 4: David Kolb’s Experiential Learning Cycle.
When a teacher is confronted with a problem in the classroom for which there is no immediately obvious solution they will investigate the problem further gathering evidence. Reflecting on what they have learned, they will reach conclusions or hypotheses about how best to deal with the problem. Teachers then implement these suggestions and record the results. If the hypothesised fix hasn’t worked then this requires further reflection and so the process continues. Like the relationship between scientific hypotheses and observations in the scientific method, and unlike didactic instruction, in experiential learning there is a dialectic relationship between between understanding, practice and problem whereby through practice the practitioner develops their understanding of the problem.
Fig. 5: The role of diagnostic/dialectical analysis in metacognition and developing a growth mindset.
While I argue for a dialectic approach to knowledge acquisition in my conclusions below, it seems intuitively true that dialectically, the abilities at different ends of Bloom’s spectrum are reciprocally reinforcing. Critical analysis appears to involve the student establishing relationships in a dialogue (dialectically) between their understanding of the world and the problem or object of analysis, using that knowledge to address the problem and learning through the process. Judgements follow from this dialogue, contribute to our understanding and can direct further investigation. Finding solutions that aren’t already given to us can be illuminating, offering new perspectives from which to view future challenges. So long as part of our background knowledge and diagnostic skill involves the ability to critically evaluate these new perspectives so that they do not become a burden on our intellectual development then they become new tools expanding our capabilities further.
If we re-examine traditional accounts of critical thinking skills we can see how the activities they describe can be brought under the yoke of critical analysis and deployed to assist in such learner transformation. Such accounts often struggle to explain how the activity works in the relationship between knowledge and evaluation, and they tend to overcomplicate the task by equating it all with the most sophisticated thinking activities. Often they also overlook the significance of the problem or intentionality in critical thinking tasks. However, I have found that our students can engage in more basic analytical activities before a more sophisticated reflective analysis of their thinking, and in doing so separate out the different tasks into an order that makes the relationship between them coherent and their role in the learning process much clearer without confusing the activity with a melange of technically demanding and daunting processes from the outset.
For example, when Richard Paul and Linda Elder promote Socratic questioning as a form of critical thinking they state that ‘[u]sing analytic questions in Socratic dialogue is foundational to comprehension and to probing reasoning. To analyse, thinkers break the whole into parts because the problems in the ‘whole’ are often functions of problems in one or more of the parts. Success in thinking depends on the ability to identify the components of thinking by asking questions focused on those components’ (Paul & Elder, 2007a: 36). This I believe is another example of what happens when detailed, critical, and reflective analysis are conflated, without appreciating that the former and latter are in reality required to improve knowledge and comprehension and the second is specifically directed at a problem; comprehension grows through a process involving critical analysis, and this process can also utilise Socratic questioning, but critical analysis is always directed with intentionality at a problem, and the learning that takes place around critical analysis takes place dialectically through interaction with this problem.
Returning to Stella Cottrell’s student study guide, once she has identified ‘critical analysis’ as the ability to determine strengths and weaknesses, to identify why they are strengths or weaknesses and reach a subsequent evaluation, in her practice activities she appears to equate it with the ability to analyse something, identify its separate parts and their functions, dismiss that which is inappropriate to focus on in order to reach an evaluation and focus only on that which is appropriate, e.g. underlying reasoning, conclusions, and evidence (Cottrell, 2005; p.58-60). This use of the term appears to fit precisely the intuitive definition I offered at the start of the essay; ‘an analysis that culminates in a greater critical appreciation of the object under investigation, where ‘critical appreciation’ stands for capacity for evaluative judgement’. However, what both appear to under appreciate is the student’s capacity to identify that which is relevant to a problem. While Cottrell recognises that we naturally have this skill and often take it for granted (ibid; p.4), her advice focuses on adopting sophisticated analytical techniques in order to extract that which is relevant.
Similarly, in a recent guidebook on achieving A grades at A-level, the tutor conflates detailed and critical analysis; ‘By analysis, we are trying to present some relevant information that is developed so that it fits the argument in the right place – it is like moulding a piece of playdough into a particular shape. The playdough is still the same piece [detailed knowledge], just developed in some way’ (Waterfield 2019: 43). In this advice the student is told that critical analysis is ‘relevant’ detailed analysis, but that the skill for which the marks are being awarded is evaluation and ‘analysis’ is necessary not so much in order to achieve it but to help others make sense of it. This gets closer to the idea that critical analysis involves identifying and deploying details that are relevant to the problem, but in failing to make this point it needlessly confuses detailed and critical analysis.
What I am arguing to be a confusion appears to arise because of an insufficient distinction between detailed analysis, critical analysis, and critical thinking or reflection. Through critical activities like Socratic questioning students can (and indeed should) return to the task of reflective detailed analysis when scrutinising their own thinking and its relation to the problem at hand, but it must be recognised that these activities are subordinate to more basic and intuitive critical activities in which the student identifies that which is relevant to the problem. Once students have used their knowledge to develop their initial responses to a problem, they can then use critical thinking tools to refine these responses.
Initial detailed analysis, problem-based critical analysis, and reflective detailed analysis functions in a dialectic relationship whereby the student reappraises their knowledge in relation to the problem. However, we can argue that these activities orbit and act in service of the primary tasks of critical analysis because problems can be addressed using this skill alone.
From the outset of my research I found that students could easily engage in analytical and evaluative thinking and writing if they were shown clearly how it differed from descriptive writing. My early resources attempted to make this distinction clear by giving students examples of these types of writing:
- ‘The issue with α is β. This is because θ’
- ‘The most important thing to note here is α. This is more important than β because θ.’
- ‘β relates to (the issue/problem/challenge in the question) because…’
- ‘This is different from α because β.’
- ‘This is a case of α rather than β because θ.’
- ‘In contrast, when we examine α we see that β.’
- ‘The criticism that α is focused on the β part of the theory/argument/issue. This is because θ’.
- ‘The strongest case that can be made for α is β. This is because θ.’
- ‘However, it is relatively weak to conclude that α as β.’
- ‘The best reason for claiming that α is β because θ’.
- ‘In conclusion the best argument for α is β because θ’
On the basis of detailed analysis alone, students can identify/highlight/emphasise what they perceive to be the most salient points in relation to an issue or problem and distinguish these from less important points. It is possible for students to discern what is relevant to the problem without using the more demanding skills with which critical analysis is confused, but if subsequent phases of learning utilise these sophisticated thinking activities it opens up far more opportunities for the dialectic growth of student thinking. Detailed background knowledge will suggest which solutions are more effective and these can be incorporated into initial conclusions which can then be tested in reflective phases of learning in which students return dialectically to reflect on the quality of this knowledge.
Students can and should engage in immediate critical responses as a foundation upon which they can build more sophisticated critical awareness. Critical thinking and more detailed reflective analysis should then be incorporated into the process of student development; it refines initial intuitions and speeds up the process of dialectic development without relying on marking feedback. Indeed, without such learner centred questioning an over-reliance on teacher led, initiate-respond-follow-up (IRF) questioning will require teacher feedback to take the place of such critical thinking. When students do not respond to such feedback the dialectic process of refining engagement with problems is sometimes not engaged in, perhaps once during green-pen work or twice if we also incorporate peer-assessment; there is too little of it over too long a period. In fact I would argue that even after more than twenty years of AfL progress such questioning has not been incorporated as fully into teaching as it could have been because we have lacked a framework within which it clearly fits and performs a coherent function.
The more complex critical analysis found in critical thinking and Socratic questioning appears to operate in the service of the more intuitive analytic tasks (identify, relate, highlight, emphasise, compare, contrast etc) that students should engage in immediately following a period of learning. It should help our students to move beyond their initial intuitions or understanding to more sophisticated thinking and reasoning. A clearer understanding of critical analysis helps provide a bridge between knowledge and understanding and the questioning that develops students’ critical awareness as these things are commonly understood. Having engaged in a more complex analysis, which with planning will force students to engage in a more detailed analysis of their own understanding, students will return to the more basic analytical tasks but with more detailed knowledge and more sophisticated awareness of the problem, as it is those tasks that will compel them to re-address the problem in the question.
Paul and Elder do recognise the role of the problem in some analysis and add that ‘[a]ll thought reflects an agenda or purpose and cannot be fully understood prior to understanding behind it’ (Paul & Elder, 2007a: 36), but they do not afford the role of the problem or the intentionality required to address it the same centrality that it adopts within a dialectic (this feature of learning is explored when discussing Vygotskian dialectics below). Accounts of critical thinking like this make them sound like games that can be played to expand the intellect and achieve ‘excellence of thought’, much as one would play sudoku. Their account of Socratic dialectic lacks the intentionality present in Socrates’ own dialogues which were always directed at some problem like ‘What is justice?’
This is not to say that analytical skills can not or should not be practiced in this way, but I would suggest that neglecting the role of the problem isolates it from its context in the developmental process, robs the activity of its efficacy and meaning, and more importantly it makes the process appear unnecessarily difficult and ambiguous, akin to learning a craft or a new language. Mirroring a theme that runs throughout texts on the art of critical thinking, Paul and Elders argue that ‘Socratic questioning employs the tools in framing questions essential to the pursuit of meaning and truth. The goal of critical thinking is to establish an additional level of thinking, a powerful inner voice of reason’ (Paul & Elder, 2007a: 36). This makes it appear that students must continually ask themselves sophisticated questions of their learning if they are to learn anything, as if they must become like Socrates himself, whereas in my research I have found that students are quite capable of identifying information of relevance to a problem, of explaining that relevance, and of relating this information to other aspects of their learning (the very first activities they must engage in when addressing an evaluative question), and are therefore more than capable of critical analysis before even considering more sophisticated thinking skills. By underestimating these first steps in moving students from lower to higher order thinking we confuse process with artistry, make the process of intellectual development appear to be a kind of middle-class pursuit accessible primarily to those students who came from families that discuss issues critically, and we rob students of their independence by making them over-reliant on subject authorities, tending towards questions like ‘Have I analysed this effectively?’, or ‘Is this the correct answer?’, rather than independently asking ‘What have I learned or do I need to learn that is relevant to address this problem, and how can I use that knowledge to address it?’
In distinguishing between detailed and critical analysis I would obviously not seek to undermine the significance of the former in the role of problem solving or development of student skills. We should teach our students critical thinking skills, but equipping them with an understanding of their own intellectual development in a way that they can take with them to address future challenges places a greater emphasis on the prior development of background knowledge. Focused detailed analysis producing strong background knowledge can pre-empt having to return to one’s learning during learner-centred questioning and scrutiny as one’s initial responses are likely to be more informed. The greater our background knowledge the more tools we have available to use to address the challenges we face. Critical thinking and detailed analysis of our thinking serves to refine that knowledge. Vacuously this says little more than that learning expands our horizons, but it is also the basis of meaningful, deep learning which takes place dialectically through the diagnostic use of knowledge when faced with challenges. Knowing this could play an important role in students’ metacognitive understanding and help them toward the development of a growth-mindset; they should expect to misdiagnose problems in the process of dialectically learning to diagnose them effectively.
Effort undertaken in detailed analysis of subject content early in the learning process takes pressure off students when they are subjecting their initial responses to problems to critical scrutiny; many erroneous conclusions can be avoided by paying more attention to what others have said on the issue. It also plays a central role within critical analysis, albeit one that is subservient to other activities, and so it has a key role within effective, learner-centred teaching.
In addition to this, as I have highlighted in the employment examples above, in real-life scenarios a diagnostic analysis would have to be preceded by a detailed analysis just as students who were presented with learning or a problem that was genuinely unique to them they may also be required to ‘inspect’, ‘examine’, ‘explore’ and perhaps ‘experiment’ or ‘test’ (detailed or primary analysis). However, as in real-world examples I would suggest that this should take time as it plays a role in the building of a knowledge base. If students in these fields were presented with an original problem and then expected to produce an evaluative judgement which offered solutions then the task would become very challenging under timed conditions. This would require a great deal of prior learning, ability, motivation and independence, and while a summative exam setting may seem like the perfect place to test a student’s capability to engage in such original problem solving, they are rarely if ever actually required to do so in exams and this is probably for the best; the pressures of analysing our way out of an escape room may be a great deal of fun, but when we have a significant investment in the solution, e.g. when we take cars to a motor mechanic, our bodies to the GP, or a potential qualification in the balance, it is exactly those head-scratching responses that fill us with trepidation. In education this really requires the type of reflective descriptive and diagnostic analysis reflected in active research or Kolb’s experiential learning cycle (fig.4), and perhaps is better representative of coursework subjects in which a product is produced over time. While PBL is criticised for not giving enough emphasis to the importance of knowledge acquisition in problem-solving (a debate covered in the theory and conclusions below), when the learner faces original challenges its focus on formative assessment appears to be more appropriate than the summative assessment that appears to form the raison d’etre of education in the main. If we are getting students ready for the world of work they should perhaps be receiving more experience of such challenges.
I have found that there are a whole range of additional classroom activities and learning experiences that can be unlocked by bringing greater clarity to the distinction between detailed and diagnostic analysis, and associating the latter with the process of evaluating solutions to problems. However, using these activities to improve students’ engagement with subject-specific problems also helps avoid problems created by an over dependence on writing frameworks. When students are encouraged to engage in the cognitive activities of analysis and evaluation and then to write them up, I have found that their responses reach another level of quality and sophistication.
I will also argue in my conclusions that a dialectic picture of knowledge acquisition such as the one presented here makes more sense of learner centred rather than outcome centred learning. Problems change, the learner’s environment, interests and motivations change. The learner however is a constant throughout and becomes the focus of the learning process. Static, dualistic models of knowledge acquisition, in which the learning process fills students with knowledge so that their consciousness correctly mirrors the world, are biased towards output, lead to over-dependence on teacher feedback and make it difficult for students to understand how to adapt their prior learning to new situations. As in the case of Paul and Elder above, teachers tend to underestimate the capacity to adapt old learning to new problems via thinking skills, and without confidence in their skills students become dependent on teachers’ affirmation that they are using these skills correctly. If the Dunning-Kruger effect is correct (that for a significant proportion of our intellectual development we lack confidence in our abilities as we develop them) then it is little wonder that an over-reliance on the artistry of thinking has produced students that lack independence and subsequent confidence. Rather than stressing the repetition of knowledge that has already been acquired by others elsewhere, the model being promoted here encourages learners to use it diagnostically to acquire and develop their own knowledge, making them more independent and adaptable to change. Students will still need to know if they are using their knowledge correctly, but there is plenty of guidance on flawed reasoning that students can, with ‘wait time’, apply to their beliefs and that do not need to be internalised in order to be used correctly. With a metacognitive understanding of how to use their knowledge to address problems in exams and beyond, and with guidance on knowledge acquisition and their own cognitive development, they will at least feel confident that they know the process by which they should be addressing problems and learning from them and can develop beyond the classroom independent of others’ praise and reassurances.
Incorporating this approach toward critical analysis into one’s teaching can unleash teacher creativity as we come to see the opportunity to use and develop many more resources than we had access to previously. The next section of this essay will explore how these can be incorporated into the analytical phases of learning within the learning cycle of each topic of study and how they transform the teacher’s understanding of each phase of learning and their subsequent relationship with their students. However, there is a tension that exists between traditional subject curricula and problem-based pedagogies that many advocates of PBL regard as an irresolvable dichotomy. In promoting the diagnostic function of critical analysis I will argue for a resolution in the conclusive section of this essay.