How does a Question Level Analysis fit with the draft Ofsted Inspection Framework, which will be in use from September 2019 onwards?
Draft Ofsted Inspection Handbook
The draft Ofsted Inspection Handbook says that:
170. When used effectively, assessment helps pupils to embed knowledge and use it fluently, and assists teachers in producing clear next steps for pupils. …
171. Inspectors will therefore evaluate how assessment is used in the school to support the teaching of the curriculum…
So therefore, whatever assessment you use has to start with taking an overall view (for example, how many pupils passed or nearly passed), but also at the important topic strands, seeing how the school is doing, and “producing clear next steps for pupils”.
Were there questions that your pupils didn’t attempt? Was this significantly higher than the national average? Was this due to deficiencies in exam technique, or does this illustrate gaps in pupils’ knowledge?
172. …The report of the Teacher Workload Advisory Group, ‘Making data work’, recommends
- that school leaders should not have more than two or three data collection points a year, and
- that these should be used to inform clear actions.
Therefore, whatever assessment you use must be effective. It must be capable of giving you clear actions through multiple analyses. For example, looking at individual topics, individual questions, cohort types and individual pupils…
…and seeing how they are progressing compared with their National Averages for their Key Stage 1 Prior Attainment.
Based on paragraph 173, however many collection points you are using, you should have:
- clear reasoning for what interpretations and actions you are going to use,
- you should have the time that is need to set assessments, collate analyse and interpret the data, and then
- act on the findings.
Interpreting the data need not be done by the school alone. You can get outside assistance from third Parties such as ourselves.
You might find that, on analysis, there is one particular question – for example, 8827 divided by 97 – which is a particular problem to your pupils, including how to tackle this sort of approximation. This analysis can be used to improve your curriculum for the benefit of your pupils.
It might also show certain areas where your pupils are doing better than the National averages, perhaps in areas which could be extended for your most able pupils, or perhaps allowing time to be diverted onto other topics.
Based on paragraph 179, if you are doing assessment, Inspectors will ask at your next inspection what you are drawing from the data and how that informs your curriculum and teaching.
You need to look more than at your school overall or the overall score per pupil. For example, maybe when you look at your girls, you find that fractions are currently a problem, as evidenced by the results of certain questions compared with the National Average, and compared with your boys’ answers to the same question.
There may be more than one way of seeing this – for example, using RAG-rating charts, you can compare the colours of results for one question compared to another, and for one cohort to another.
The percentage of questions not attempted, when compared with the National average, might also indicate improvements for your curriculum.
Amanda Spielman’s comments
Amanda Spielman, in her speech of 16 March to the 2019 ASCL annual conference, said that, what inspectors may be looking for if your previous results were disappointing, were your actions:
- They would be looking to see how you have started to make your curriculum more coherent,
- How you were increasing your teachers’ subject knowledge, and
- How you were assessing and refine your curriculum and teaching as you go.
You will therefore need to look at the outcomes in your assessments, looking at where your school needs attention, like measurement or “ratio and proportion”, and how this can be impacted into future lessons.
However, you should also bear in mind the requirements of the SATs, and the weighting of the topics, to ensure that your teaching is effective, and not leading to huge improvements in an area which has a low level of importance in the assessment.
How will inspectors rate your school? What will they take account of?
One area in the draft Ofsted Inspection Handbook under the “quality of education” is that:
- In outstanding schools, pupils consistently achieve highly, particularly the most disadvantaged.
- Whereas in inadequate schools, the progress that disadvantaged pupils make is consistently well below that of other pupils nationally and shows little or no improvement.
Therefore, you need an analysis that doesn’t just look at boys/girls, but also looks at your disadvantaged pupils. For example, how does your disadvantaged pupils’ results compare with their Scaled Scores had they got neutral progress?
But you don’t want it just for disadvantaged pupils. What about:
- your Low/Middle/High Prior Attainers,
- pupils who have come into your pupils recently (“Mobile”) or those who have been there for several years, and
- pupils with SEN or who have English as an Adjudication Language?
Accord to the Ofsted Inspection Handbook, in schools which are good or outstanding:
- Teachers and leaders use assessment well, for example
- to help pupils embed and use knowledge fluently,
- or to check understanding and inform teaching.
So you might look at individual pupils and see how they are done. For example, Jane might be doing 12% better than the National Average for Punctuation, but is doing 13% below for Grammatical terms and word classes. With that knowledge, you might want to investigate why that is the case.
Ofsted will be looking at your Intent, your Implementation and your Impact:
- The school’s curriculum is coherently planned and sequenced towards cumulatively sufficient knowledge and skills for future learning…
- The work given to pupils is demanding and matches the aims of the curriculum in being coherently planned and sequenced towards cumulatively sufficient knowledge.
Therefore, your curriculum should be backed up with the occasional assessment.
- Pupils develop detailed knowledge and skills across the curriculum and, as a result, achieve well.
And for this, you should look at your school as a whole, for various cohorts, and for individual topic strands and questions to see what could be improved, and how your curriculum could be developed.
If you want our top 10 tips in choosing a good Question Level Analysis, then please have a read of our top 10 tips article.
If you want other schools’ experience, please read our article as to how a good Question Level Analysis helped them.
Don’t forget – you can do a Question Level Analysis:
- of your pupils who have recently done their SATs,
- of your current Year 5 pupils late in the Summer term, and
- of your current Year 6 pupils in the Autumn or Spring term.
And you can use the 2016, 2017, 2018 or 2019 mock papers.
For more information about our Question Level Analysis services, please click on this link.
If you need any assistance in this, or want to know how we can reduce your teacher’s workload by doing the analysis for you, please contact us by clicking on this link.
Thank you very much for reading this article, and we look forward to helping you further.