4. Indicative Findings
To illustrate the contribution of the methodological approach to a policy field, some of the areas in which a distinctive contribution has been supported are outlined below. In quotations, ‘By6L1’, for example, refers to Study B year 6 learner 1, ‘DT3’ to study D teacher 3, ‘Cy8LO’ to study C year 8 lesson observation, etc. Almost all quotations are from mid-year focus groups (or for teachers, interviews), since those often offered the most direct communication of participant views; other sources are indicated as ‘LO’ as above, or as ‘survey’, or with documentary source. We address each of the selected emergent themes in turn, referring to cross-phase evidence patterns as appropriate: there were a variety of cross-phase commonalities and differences identified, together with variation within and across schools. At a high level, we show how the longitudinal approach exposed issues not only for the inception, but also for the sustainability, of curriculum system coherence. We evidence ways in which the habits, skills and dispositions of teachers and learners interacted with the focus curriculum and assessment materials over time, including as those materials changed. Together, these framed the emergent meaning-making and practice at classroom level.
4.1. The Learner as Curriculum Policy Actor and Curriculum Maker
School learners are commonly conceptualised, at least implicitly, as passive ‘receivers’ of curriculum, and our data supported that in some classrooms. However, the vast majority of our participant learners of all ages (at least 90%) appeared willing and constructive (though sometimes constructively critical) participants in their mathematics education. Learners very frequently made choices among the texts available to them at some point in the lesson observed. Even in primary classrooms, we almost always observed at least some learners choosing between, and selecting from, teacher or class discourse, front-of-class slides, textbook or practice book presentation, or peer learner, as their first external reference point for independent work. For almost all sample learners to around age 15, the dominant text, and source of mathematical authority, appeared to be teacher discourse rather than curriculum or assessment materials, though in at least 70% of focus groups across studies, learners identified particular features of written texts that spoke to, engaged and excited them:
I like how you can choose, …what method you want to do, and explain why. If someone (in the book) says something like this, and then someone says something like that, you can choose the easier method to work out the answer (By4L1);
Always after the challenge there tends to be a Reflect. I think they’re very useful, because once you’ve done your work, sometimes I feel like I need a bit more thinking about it so I don’t just forget about it (By4L2);
I really like the Challenges. They’re quite fun. They’re like, although sometimes it’s hard to explain things, I actually like quite a bit of a challenge (By4L3);
The Reflect is the hardest one so that's why we do it together (By2L1); And sometimes the Challenge is really hard too (By2L2); Yes, because Challenges are meant to be very hard (By2L3).
As learners approached GCSE examinations at age 16, though, and particularly in Spring of year 11, assessment texts and narratives came to dominate:
A small question with a lot of marks, so it gives you a hint it's going to be a difficult question (DL1);
So now that, because I’m getting 50% on most of (the papers), and that’s a 4, so I know that if at least I’ve done half of it, then I’ve got a 4, which that means that that extra for a 5 is kind of just that 10% more (EL1);
I find I just like to build my confidence. Because if I'm struggling with one of these (early questions), I'd be thinking, how am I going to pass if I'm struggling with the first few? (CL1).
This was often abetted by teacher performativity (Ball, 2003) approaches, evidenced in both classroom observations and teacher interviews in both year 10 and year 11:
We think that's the way to go to just start preparing them a bit more for common questions (DT1);
I’m trying to get them to get the first 10 questions, I’m trying to get them to get them just 100%, they make some stupid mistakes, it’s unbelievable, so I’m just trying to get them to try not to rush it, I’m still trying to do that mark a minute thing (DT2).
This led more reflective learners (at least half of focus group participants, across demographic groups) to adopt a ‘curriculum narration’ role, making active choices among a range of examination-focused resources – but also often adopting ‘constructive critique’ [
20] of curriculum, resources and enactment:
I think the exam pressure sometimes gets to a learner, so if you kind of have things in bold, it… I've done many questions where I think I'm answering the question, and I look back at the question, and I've answered something completely different. So I'd have to cross it out and restart. So I feel like bold would draw my attention better to what I need to do (DL2);
If you want the maths paper to be like strictly about maths and learn how to do it, then bolding it... But if you want it to be more about problem solving and like picking up what you want to do then obviously not (DL3);
There’s 160 people in our year, and they’re putting us all under the same lot of pressure to do one style of exam…And everybody has different learning styles and they want 160 people to sit the exact same exam and get the grade…if they’re sitting us all in the same exam people will thrive off certain questions (EL2).
There was, though, considerable resistance among mid-to-lower-attaining GCSE learners to some examination-linked experiences that were perceived as oppressive and ‘miserable’, particularly in retrospect:
I put so much pressure on myself to pass and I was just stressing the whole time. I need to pass, I need to pass. It’s just not a nice situation (DL4);
Years 10 and 11 were just miserable: there is so much to know, and I wasn’t succeeding with the papers, it got more and more stressful so in the end I tried to avoid them (DL5);
Some of the worded questions in the exams don’t always relate to experiences I’ve had. I can do the maths in lessons, but some of the exam questions seems designed to wrap that up so that I stress about what maths to use and then I can’t show what I can do (DL6).
A few made suggestions as to how to allow learners to show more of what they were capable of; one was to include course work ‘maybe like having some coursework like added to the mark’ (DL8) or ‘more multiple choice questions’ (EL3). About 50% of learners in affected cohorts could articulate advantages of engaging with authentic or familiar, realistic contexts, although a further about 25% were sceptical about such attempts:
Like the fish tank with the sand in it and then you have to convert the volume into meters, I think that’s quite a good question to have in there. Because that’s sort of almost a real-life situation, not many of the questions are (CL2);
Some of the questions are just silly, and that winds me up: whoever heard of someone buying 150 watermelons? (CL3).
A range of learners commented on the value to them of provided, or self-initiated, diagrams:
I think the diagrams help because, for me, if there's not a diagram I always draw … a diagram, or … a graph or something, so that … I understand it more than just the words… It's hard to visualise in your head without a diagram (CL4);
I think maybe for certain questions more diagrams could be useful because I know personally that sometimes I work better when I have the thing in front of me. For ones like probability where you have a certain amount of coins or balls … having that in front of me would definitely help. But for others, I think it’s fine (CL5).
- -
although they were also impatient of diagrams that might mislead:
There was a question yesterday in our paper …it was like a diagram but, and the angles were different sizes, but the smaller angle was bigger, the big angle smaller. But it made me think I'd done it wrong, because on the sides of the diagram it didn't say whether it was drawn accurately or not (CL6).
Importantly, learners who had experienced, or were experiencing, a range of other approaches to summative assessment during the pandemic were quite sophisticated in their analysis of the affordances and constraints of those, again functioning as curriculum narrators:
Perhaps use a mix: exams are the only really fair way, but they could perhaps have a contribution from teachers, to incentivise hard work and give students confidence that not everything depends on that eon day (EL4);
(Centre-assessed grades were) completely unfair in some ways, there should have carried on with exams, in a socially distanced manner or invigilated online - grades have to be earned not given, if they’re going to be respected (EL5).
Such issues are further explored in [
30]. More broadly, the majority of 14-18 year olds, and up to about 30% of younger learners made relatively independent and selective use of the funder’s and a range of other resources. These included freely-available digital curriculum apps and online fora they chose for complementary mathematics support or enrichment:
Different online resources are useful for different topics, depending on how secure I am, I can choose what works best to improve (CL7);
Looking back, I’m now much more selective of whether I use apps or an online room: there are places that are good for consolidation and others that really help if you want to know a bit more, beyond what the exam needs (DL4);
I can rewind videos teachers record and learn at my own pace’ (Ey12L6);
I created a study group where we would all join a Teams meeting and do maths questions and topics…. (That) helped all of us in revising and maintaining productivity (Ey13, L7).
These students, who crossed demographic and prior attainment boundaries although were dominated by higher-attaining students, could be considered to act as ‘narrators’ of curriculum. Some students also functioned in a variety of other policy roles, the weakest-attaining quartile often as ‘survivors’ or ‘critics’, but another about 10%, particularly those supported to attain highly at either ‘tier’ of entry, as ‘translators’ or ‘enthusiasts’ in Ball et al.’s (2011) terms. Importantly, analysis of learner-sourced data in each element of the study led directly to changes in funder curriculum and assessment materials. It also fed into evidence underpinning central (Ofqual) guidance that developed over time and resulted in key changes to GCSE and A Level mathematics papers. Details are given in a range of funder-internal reports, but changes included use of more space and simpler language in younger learners’ books; enhancement of quality and quantity of problem-solving in resources for 11-14-year-olds; accessibility and structure of examination papers at age 16; enhanced range pf support resources for A Level Mathematics preparation. Additional, strategic and cross-cutting impact has included greater central exemplification of intended curriculum changes, and central addressing of specific teacher development needs identified[
17]. Such changes are especially important in a system where direct use of student voice ha s historically been accorded only a very low profile.
4.2. Teacher and Learner Lenses Compared
Throughout phases, lesson observation, teacher interview and student focus group data taken together showed fairly close correlation between teacher discourse and that predominantly presented by their learners, although the ~5000 learner surveys showed small-scale (up to about 10%) contra-discourses evident in almost all study classes, underlining the importance of large-scale as well as more in-depth small-scale data collection, and possibly also, privatised, as well as shared, narratives. Teachers who embraced curriculum intentions at classroom level almost always sought resource support for doing so, but functioned as active agents in developing enactments for their classes:
(The teacher) took some of the structure out of the question, to enhance the challenge; she later explained she thought they could cope with a harder question than in the book, and wanted them to experience getting stuck – as well as having more than one possible solution (Cy8LO).
Their learners largely talked positively about challenge, identifying aspects of curriculum/assessment-related materials that supported that, and often offering proposals for further resource development:
The book is pretty good, but the question we had today was harder than what was already there. It took several of us to work out how to do it, but I think that’s good – the book does encourage you to work together, but perhaps if there were starred questions that were particularly challenging, that would be even better. Those can be really rewarding, but it’s also good if you feel OK if you can’t do it’ (Cy8L8).
Other teachers, in contrast, sought ways to ‘survive’ in the new curriculum and that too was largely mirrored in their learners’ discourse – in all phases:
Since the new practice papers have been published, we don’t do the reasoning questions in the book. In principle, I was in favour, but my students found them really hard, and if they’re not going to be in the exam, it’s not worth the effort (Cy10T1);
We don’t do all the questions in the book, because the reasoning ones, and some of the problem-solving ones, are too hard (Cy10L9);
A lot of learners are finding it incredibly difficult to access the material. So I give them more structure and hints, and train them to recognize where they can pick up a mark or two – we don’t attempt the real question (Dy11T3);
We’re doing OK: Miss tells us what words to look out for, and where we can pick up enough marks. We can usually get enough (for the target grade) without having to go near the last few questions, provided we are careful to get what we can earlier in the paper (Dy11L5).
Analysis of teacher and learner talk, as above, exposed the hegemony of assessment materials, particularly at upper secondary level, and it was here that the alignment of teacher and learner responses to curriculum change was most obvious. For secondary cohorts 3 and 4, if teachers adopted a procedural approach with learners expected to achieve only poorly on papers, that focus appeared transactional and their learners generally presented as receivers, although there were exceptions evidenced in every study class; peers matched for prior attainment but entered instead for papers on which they realistically expected to do well, normatively appeared to thrive, seeking peer or material support and talking confidently about using mathematics post-16:
We shouldn’t be entering them for papers they can’t be really successful in, just so they might get a higher grade: … I think that’s child abuse. I’d rather they got a better appreciation of the maths they can do and not the maths they can’t (Dy12T4);
I only did the Foundation papers, but I did quite well on those, and now I’m studying Core Maths, there are some new ideas, but that’s OK, because they’re quite useful for what I want to do, and I’m getting more confident I can make sense of some of the harder ideas (Dy12L1).
At A-level, surveys showed similar initial relationships between teacher and learner perceptions. That was exacerbated when successive iterations of the new assessments, especially around problem solving, posed what several teachers called ‘moving goal posts’ [
14]. Where (about half of) teachers were confident to attribute such change to the challenges of assessing new aspirations, their students of all demographic and previous attainment were generally phlegmatic about the varying demands they encountered in emerging papers:
It's a bit challenging, since we can’t be too definite about expectations with the students. But I just tell them, focus on understanding the maths, and then show the examiners what you can with with whatever they choose to ask you (Ey13T3);
They seem to be changing their minds about just how hard they’re going to make the problem-solving, but that mirrors life and it’s all good experience. In a sense it doesn’t matter how difficult they make it on the live papers, we’ll all be in it together and whatever, it does make you think with your maths (Ey13L26, same school, accompanied by nods from peers).
Where, on the other hand, teachers felt reported, or communicated, feeling oppressed or overwhelmed by such demands, students usually seemed unconfident to deal with such variation: ‘We now have no idea what to expect, which really doesn’t seem reasonable, and it does mean it’s not worth investing too much in getting to know the new specs’ (Ey13T7); ‘We’re now really worried because we don’t know what they’ll be asking or if we’ve prepared in the right way’ (Ey13L42, same school) – although again, there were exceptions to that, alignment, for example ‘The teachers aren’t sure what will be wanted, but I think you just have to get on and deal with that uncertainty: we only get the one chance’ (Ey13L45, same school).
As the A Level study continued learners appeared to retain confidence in their teachers, but many also over time developed greater proactivity in independently selecting materials and strategies to match their emerging perceptions of assessment demands. They often framed such actions in terms of the ‘exchange value’ of the qualification, for example its use in supporting access to their target university course, whereas teachers’ concerns were more focused on learning the mathematics, including for future use, apparently less conscious of the qualification’s role as gatekeeper to future career pathways. It would appear that while assessment systems in themselves might be insufficient to drive challenging curriculum reform within a high-stakes context, they are harnessed and influential in myriad ways.
Data showed older learners generally more confident, and better equipped, for proactivity in meeting learning needs via peers, older friends, siblings, and digital resources, and divergence from teacher recommendations. A-Level learners also showed other divergences from teacher standpoints. They perceived mathematical demands in their courses progressed fairly smoothly from pre-16 learning, but a step change in workload; given revised curricula, their teachers in contrast were often still struggling to accommodate new expectations of subject and subject pedagogical functioning.
Taking these two areas together, we see many learners were themselves active in policy interpretation; they also significantly influenced both teacher and edu-business implementation: the study therefore supports a fairly nuanced conceptualisation of learner as mathematics curriculum policy actor.
4.3. Edu-Businesses as Co-Constructors of Curriculum with Teachers and Learners
The funder edu-business as policy actor influenced implementation in all study classrooms via their curriculum and assessment resources, and as above, those resources served as narrators, enthusiasts, and sometimes entrepreneur of mathematics curriculum. For both teachers and learners, the textual hierarchy, in this high stakes assessment environment, was dominated by their published examination papers for ages 16 and 18 as classes approached those examinations. However, the cross-phase nature of the study revealed that for our sample, the dominant edu-business role for about 80% of our primary teachers – and at least half of primary learners - was as highly influential narrator, sometimes perceived as enthusiast, via their curriculum materials.
However, that did not mean that all teachers in a school responded to the new curriculum, or used the focus funder resources, in the same way. The school mathematics lead/Head of Department was observed very often to act as lead policy actor, whether as narrator, enthusiast or less positive role – and was typically influential on practice at least partly via the school mathematics ‘scheme of work’: ‘So when I’m planning, I start from the scheme of work our maths coordinator put together, and I try to stick to that in principle because then the children get reasonably consistent experiences over time, but I tweak it to me, and the particular children in my class’ (A17, T5). However, analysis of lesson plans and observations, as well as teacher interviews, showed individual teachers within a school responding differentially, one with perhaps minimal compliance and another with deep investment in getting to understand, select from and adapt materials for their own learners. The longitudinal nature of the studies meant that we could observe teacher change in practice over time. Almost always, for the about 50% of teachers who showed significant progress towards the intended curriculum practive over the course of the studies, that appeared to require considerable and sustained (at least eighteen months’) investment in rethinking practice with the support of at least the focus materials.
That was true in a fresh way during the pandemic, when the new constraints widely catalysed a re-thinking of primary teacher curriculum priorities and inter-dependencies in which curriculum materials, and particularly teacher guides, appeared to support transformational change in subject and subject pedagogical knowledge and practice for about a quarter of participating primary teachers:
The teachers were saying how much they’ve gained from re-thinking their planning … for doing it online, using the Teacher Guides: they’ve come to look at things in new ways and had to sort out the main points, so they’ve now got a better structure, and better priorities, in their heads. And our formative assessment has become absolutely central to our practice: are the children bringing what they need? If not, then we need to nurture that before we can build on it (By4T1);
I keep getting these ‘lightbulb’ moments where the maths suddenly makes more sense, because I’ve had to go back and sort out from the materials what are the absolute essentials, what are the building blocks for that, and how will I know whether the children have ‘got it’ – and my teaching is in a different place because of that (By6T2).
Longitudinal data showed that had also been true, if less prevalent, in more ‘normal’ times. In our sample, though, that happened only when teachers invested over a sustained period, and usually, collaboratively, in getting to know the curriculum materials, and particularly, the teacher guidance. In contrast, prevailing secondary curriculum – and assessment - materials did not seem to have that role: curriculum materials acted as narrator and translator of the intended curriculum, supporting teacher roles as narrator, receiver, transactor – and sometimes, receiver of policy – but even for teachers of lower secondary learners not teaching examination classes, we saw little evidence of curriculum materials catalysing significant, and certainly not transformational, subject knowledge or pedagogical quality change over the pandemic period. This was true also of assessment materials, though as above, the ways in which, and extent to which, they assessed curriculum, did frequently catalyse change in curriculum coverage and emphasis.
Influence of course was in both directions: funder enhanced understanding of teacher and learner perceptions has already served to develop resources in ways participants think helpful. In a marketized context for both curriculum and assessment materials, that is likely to support market share, as well as to inform edu-business interactions with policymakers.