Evaluating the effects of a maths coaching programme on Year 6 pupil premium pupils

London South Teaching School Alliance

Description of the innovation

Expanding to work with primary schools, the CoachBright maths coaching programme supports middle to high prior-attaining disadvantaged pupils, ensuring they reach the higher standard in their SATs and leave Key Stage 2 (KS2) with the confidence and metacognitive skills to excel in secondary school from the word go. The programme involves university students coaching small groups of pupils through self-directed learning activities for six weekly, hour-long sessions.

This project enabled six schools to each identify two groups of up to four underachieving disadvantaged Year 6 pupils (up to eight pupils in total in each school) with high to middle prior- attainment to receive the CoachBright programme before their SATs tests in May 2019.

As part of this project, CoachBright collaborated with the South East London Maths Hub to ensure the teaching sequence their coaches used matched the mastery approach currently being promoted as best practice in mathematics by the Department for Education (DfE). The intervention was also adapted to use the Learning by Questions app, building its sequences of maths questions into the learning programme for the coaching sessions, and enabling coaches and teachers to immediately access assessment data and set relevant homework.

Summary of the evaluation

Forty-seven Year 6 pupils from six primary schools across London were involved in the intervention. A further 48 Year 6 pupils from six other, comparable, primary schools acted as controls. According to 2018–19 data, both intervention and control schools had relatively high levels of disadvantage in their pupil population. Two intervention schools and five control schools had higher than national average levels of pupils eligible for free school meals (using FSM Ever 6). All but one intervention and one control school had higher than national average levels of pupils whose first language is not English.

Both intervention and control pupils completed the epiSTEMe Maths Attitude Questionnaire pre- and post-intervention. They also sat the 2018 Maths SATs Paper 2 reasoning test pre-intervention so that scores could be compared to the scores they obtained on their 2019 Maths SATs Paper 2 reasoning test post-intervention. Year 6 teachers of intervention pupils also provided attainment predictions and outcomes data and wrote short assessments of their eight pupils’ learning, confidence and motivation in maths pre-, during and post-intervention.

Summary of findings

This very short intervention (six hours per pupil) had a small but positive impact (effect size +0.06) on mathematics attainment as measured by pre- and post-test data. Teacher assessment data appears to support these findings, since 68% of pupils met ambitious targets set by their teachers, with 6% exceeding their target, and 62% of pupils exceeding teacher’s predictions of their likely attainment without the intervention. However, control group teachers were not asked to provide teacher assessments, target data or descriptions of pupil progress so it is not possible to know how similar pupils who did not receive the intervention progressed against these targets.

Teachers’ qualitative feedback also supported the positive effect of the project, with 64% of pupils being seen to make positive progress. Although the intervention had no measurable impact on pupil attitudes towards mathematics, as measured by the pupil attitudes questionnaire, teachers’ qualitative feedback did identify perceived improvements to resilience and confidence.

The process evaluation suggests that the intervention ran as intended, with no significant concerns about the quality of provision. Observations of coaching sessions by the evaluation team indicated only minor concerns and these were mainly around session management (eg, pace and managing diverse learning needs) and behaviour management. Coaches identified small tweaks to lesson planning and resources which were tackled mid-way through the programme in order to ensure the learning needs of less confident pupils were better met.

Limitations of the study include non-randomisation of intervention and control groups, teacher assessment data being less reliable than standardised tests and an unclear gap in terms of time between baseline and impact testing.