School improvement comes at a price, study shows

Schools which improve their results may be paying for it by increasing their pupils' levels of disengagement from learning, a major new IOE study based on survey data from more than 9,000 teenagers concludes.

The results present a challenge for teachers, say the study's authors who argue that all institutions' ability, in reality, to raise pupil engagement in learning may be limited. However, the findings suggest that schools, educationists and policy-makers are going to have to work harder in trying to find ways to enhance pupils' engagement with what they are studying, while also raising grades, says the research, being presented at the British Educational Research Association conference today.

Academics at the Institute of Education, University of London, analysed the answers to questionnaires completed by more than 9,000 pupils as part of the Longitudinal Survey of Young People in England, which has been following the progress of pupils who were in year nine in 2003/4.

Each teenager completed questionnaires when they were in year nine, in 2004, and then in year 11, in 2006. Pupils were asked whether they agreed with 12 statements as part of the survey – such as "I am happy when I am at school" and "In a lesson, I often count the minutes till it ends" – which the researchers then used to work out their "emotional engagement" with learning.

The teenagers were also asked to say whether they had played truant in the last 12 months, providing a measure of "behavioural engagement".

Results were then linked to national databases allowing analysis of test results of the pupils and their schools, how much each pupil's school had improved its results as measured by "value-added" scores, plus many background characteristics.

The study found that schools with improving value-added results tended to have less engaged pupils, under the "emotional engagement" measure. The academics questioned whether this was because of the types of pupils that schools with improving value-added results tended to educate, but found that this did not account for the pupils' levels of disengagement.

The study did find, however, that schools with improving value-added results tended to have relatively low numbers of pupils saying they had played truant in the past year.

The research concluded: "We find that pupils who are attending schools that are improving their academic achievement levels, as measured by their school value added scores, are on the one hand becoming less positive in their attitudes towards schools but equally are also less likely to have increasing rates of truancy. This is a striking result."

The results on emotional engagement may reflect the fact that schools pay a price for raising their value-added results by implementing policies which may be unpopular with pupils such as enforcing homework or increasing the emphasis on literacy and numeracy. "In the process of improving school value added scores, with increasing emphasis on test score performance, some pupils [may] become less engaged with school," the study says.

"Whilst these results potentially give ammunition to those who claim we are over testing English pupils with our league table approach to school quality, we would argue that in fact they lend support to the view that there is a limit to what schools can do in promoting pupils' engagement.

"We do, however, need to recognise that change can be challenging and find ways to engage students better with attempts to raise standards in our school system."

"School disengagement and its causes", by Anna Vignoles, Francesca Foliano and Elena Meschi (all of the Institute of Education, University of London), will be presented at the BERA conference today (Friday, September 3rd).