No legislation no bad thing

Conor Ryan suggests that the absence of education legislation in the Queen’s Speech may be a blessing for education reform

When I was a special adviser at the Department for Education in the late 1990s, it was seen as a measure of a Department’s success the extent to which it achieved legislation in the Queen’s Speech. This essentially macho test often led to more legislation than was strictly necessary to achieve policy goals.

Many people forget that although the Labour government needed legislation to restrict most infant class sizes to 30, it needed no legislation to introduce the literacy and numeracy hours in primary schools. The latter were the result of a mix of persuasion and accountability, and were arguably more effective as a result.

And legislation was too often used as a way to trumpet changes that could have been introduced less dramatically. Trust schools – the centrepiece of Tony Blair’s 2005 education reforms – were a good example. As with Michael Gove’s first academies legislation, the essential architecture was already in place, and what changes were needed could have been introduced with less fanfare through regulations.

So, it is no bad thing that there was no education legislation in this year’s Queen’s Speech. Of course, that didn’t stop the Government using the occasion from getting Her Majesty to remind Parliament of changes already in train, such as the curriculum overhaul or performance pay for teachers.

But nobody would argue that Michael Gove is any less powerful because he hasn’t got a fifty or a hundred clause bill to take through Parliament over the next twelve months. And I doubt any of his junior ministers – who would be tasked with the legwork – is overly concerned either.

However, what it does mean is that it is all the more important that changes the Government is introducing get the scrutiny they deserve, and that they are subjected to the sort of rigorous evaluation – usually through randomised control trials – that the Sutton Trust and the Education Endowment Foundation are using.

That is important not just for ministers who want to ensure that their reforms are making a difference to results, particularly for the poorest pupils, but also if they are to gain buy-in from teachers and headteachers.

With more than half of all secondary schools now having academy status, as well as a growing number of free schools and university technical colleges, schools are getting used to having more freedoms than before. And while complex legislation can be important on some issues – such as ensuring a fair admissions code – it is a blunt instrument over issues such as the curriculum or performance related pay.

Over-complexity militates against successful reform. When Estelle Morris first introduced performance pay in 2000, the intervention of the teaching unions ensured the whole process was wrapped in endless bureaucracy.

Leave aside for a moment the perfectly valid issue of the impact of PRP on attainment – though Gerard Kelly’s recent TES piece show why in this case there are other issues to consider – the real problem is that legal issues come to outweigh the flexibility that allows heads to reward good teachers in a straightforward way.  A less complex system may prove to be more effective in overcoming the culture against PRP in some schools. And we might then have some serious research on the issue too.

But those increasingly independent state schools will equally need to be persuaded on the curriculum – including on the detail now planned in subjects like history – and on other issues where ministers feel strongly. As they do so, it is important that they use evidence rather than past practice or even DFE guidance to make their decisions.

That’s why the increasing popularity of the Sutton Trust/EEF Toolkit is so important. Next week, we plan to publish new evidence of just how popular it is becoming. But in the meantime, we should reflect that giving Michael Gove and schools a break from the 2014 Education Act is not only no bad thing, it may allow the breathing space needed for genuine reform to take place.

Worlds Apart

Lee Elliot Major on the challenge of getting research to impact on education practise

As I revealed the next slide, there was an audible gasp among the 200 strong audience. I knew at that moment I had lost the crowd. Whatever I said next, everyone’s mind was focused on the big fat zero that sat at the bottom of the impact table staring out from the screen. As I turned to face the angry faces, I saw that the throngs of hard working, decent teaching assistants (TAs) had turned into a lynch mob.

On reflection, presenting the findings of the Sutton Trust-EEF teaching and learning toolkit to TAs at the end of a long hard term was not the best idea. This guide to the best (and worst) bets for improving results in the classroom shows that TAs have on average zero impact on the attainment of children. Now as I told my audience that doesn’t mean we should sack all classroom assistants. But it does mean better training, preparation and management are needed to enable the 220,000 of TAs in our schools (costing the public purse over £2 billion a year) to help our children learn.

Sadly this nuance was lost as the discussion descended into an increasingly fractious argument. No amount of caveats and constructive comments could calm the enraged ranks of TAs. All they could see was an attack on their livelihoods. I returned to London that Friday afternoon feeling like I had been mauled in a playground fight.

This admittedly was one of the more contentious toolkit talks I have given to schools during the last two years. The experience highlighted the potential evidence has to improve practise and policy, and the power a succinct accessible summary of research can have. But it also demonstrated the huge challenge of enabling evidence to actually impact on classroom practise in a constructive and useful way.

I’ve been reflecting on all this, as I prepare a talk for an Institute of Education this week on how research can impact on policy and practise.

What I will say will seem blindingly obvious, but is almost universally ignored. My ‘take home’ message is that we must acknowledge the fundamental cultural differences between the worlds of media, academe, policy and practise – if we are to reach the promised land of evidence based practise. We must recognise that communication is as an academic might say a ‘highly non-trivial task’.

Each of these worlds has its own jargon, beliefs, rules, aims. Like working with different countries, we need to embark on genuine translation and efforts from all sides to make it work.

As a former news editor, my one piece of advice to reporters was to spend as much time on the writing and presentation of articles, as gathering the news itself. What’s the point if no-one will read what you have found? I now hold the same view for the work of an education foundation: our toolkit has been successful as we spent many hours thinking carefully about how to present the often abstract and complex findings of education research.

But after years of working with schools, I’m afraid I’ve had to re-assess this rule. To affect genuine change – this is just the start: much more has to be done, and in the schools themselves. Powerfully presented evidence isn’t enough. There are countless examples of things we know work, but fail to embrace. We don’t do exercise – even though we know it’s good for us. Doctors still fail to wash their hands regularly – the most simple of medical safeguards.

For evidence-based education to work, we will need to free up time for teachers to consider research. We may need to create research leaders in every school. Inspectors may need to encourage the use of evidence more when they visit schools.

This I’m glad to say is the increasing pre-occupation of the Education Endowment Foundation as it strives to find out what works in schools. It won’t be an easy task: as with the assembled TAs during my talk, we all tend not to want to listen to evidence that confronts our own prejudices – even when the messenger has the best of intentions.

Evidence is just the start of finding what works

Lee Elliot Major and Steve Higgins, professor of Education at Durham University argue that effective implementation matters once the evidence for what works has been identified.

In England we spend around £2 billion a year on 170,000 teaching assistants in schools. The total impact of this money on the attainment of our children is zero.  The best evidence we have indicates that for every TA who improves a pupil’s progress there is another whose deployment has a negative impact.

This is a powerful example of why we need evidence based policy and practice, but it also highlights the difficulties of promoting changes to improve practice – because finding that TAs are deployed ineffectively does not tell you what to do about it.

Such issues are soon to be faced across government, following the launch last week of a network of What Works centres to champion evidence-based social policy.

At the launch, Oliver Letwin, Minister of State at the Cabinet Office, said that the biggest question was why government hadn’t done something like this before.  But if government hasn’t, others have, and the centres will be building on existing practicies of evidence use in health and education.

In health, the Cochrane Collaboration this year celebrates two decades of producing systematic research reviews. Its approach has shaped advice offered by the National Institute for Health and Clinical Excellence on NHS treatments.

The cultural shifts that introduced evidence-based medicine to surgeries and hospitals 20 years ago are now playing out in classrooms and schools. The What Works education centre will use a toolkit we developed three years ago summarizing thousands of studies to show the best and worst bets for improving pupils’ results. It is the model now being advocated by the Cabinet Office for other policy areas.

Since 2011, the Education Endowment Foundation, a charity that aims to improve the educational achievement of disadvantaged children, has developed this toolkit into a resource for disseminating evidence across the sector. The EEF has overseen a quiet revolution in England’s schools, commissioning some 40 randomized control trials so far.

The toolkit shows that working on teacher-learner interaction – improving feedback to pupils, for example –  gives the biggest bang for the education buck.  Yet our surveys reveal that most head teachers prioritise actions that evidence suggests, on average, have little impact: reducing class sizes or recruiting TAs.

In education, the route to evidence-based policy is particularly challenging, because the navigation instruments are less powerful and predictable than in medicine.  Imagine a world where the laws of nature vary through time and space. That is the reality across the thousands of different classrooms, teachers and children that make up our education system. Over the last 30 years curriculum and assessment have changed many times, and variation in schools and teachers has a profound impact on how an intervention works or doesn’t work.

 We have little idea how to help schools implement the best bets for improvement. Some may need a highly prescriptive programme; others general principles to shape and evaluate a tailored programme.

To return to TAs, for example, the evidence does not mean that they should be scrapped. There are many ways in which TAs might be better recruited, trained, deployed and evaluated. Some approaches will be more effective in lower-performing schools, schools serving a high proportion of children with special needs, or schools with particular teachers. Knowing what works where and for whom could improve a school’s choices about TAs and everything else.

A commitment to what works (strictly, what’s worked) in education must also consider the constantly changing pedagogical landscape. Take phonics teaching: if the current emphasis on phonics becomes routine, then remedial support based on phonics is likely to become less effective than research currently suggests. Children who have failed in a phonics-rich pedagogy may benefit more from a different remedial style.

These are important lessons for the other four planned What Works centres. Evidence can be boring or inconvenient for politicians more interested in an immediate and popular policy fix. But, as Letwin stressed, “this is only the start of a journey, not the destination”, and the outlook at this early stage is promising. This programme has the potential to revolutionise public policy in areas as diverse as ageing, education and policing, replacing dogma and tradition with research and randomised trials. Billions of pounds of public money could be saved.

This post first appeared on Research Fortnight

What Works – A winning formula

Peter Lampl welcomes the designation of the Sutton Trust and EEF as the What Works Centre for Education

This week the Sutton Trust was, together with the Education Endowment Foundation, designated the What Works evidence centre for education by the Government. There will be six leading evidence centres and we and the National Institute for Health and Clinical Excellence (NICE) have been selected to lead on education and health respectively. The centres will be the first port of call for advice on the latest research on the impact of Government programmes.

This is recognition of the Sutton Trust’s focus on evaluation and research in all the work it does. We have always aspired to subject our programmes to robust review. And as an independent foundation we have used evidence to challenge or support the Government’s education policies.

The Trust has funded over 120 research studies in the areas of social mobility and education. But it is primarily a ‘do tank’. Our flagship summer school programme for example is now the largest national university access scheme – but it is also the most scrutinised programme in this field.

We know they have impact: over three quarters (76%) of summer school attendees go on to a leading university, compared with only 55% of students with similar backgrounds who aren’t on the programme. We also know they are highly cost-effective: when Boston Consulting Group did a cost-benefit analysis of the Trust’s programmes – comparing the lifetime earnings benefits for the individuals on the schemes with the money spent – summer schools were among the programmes resulting in returns of over 20:1.

It was these disciplines – assessing the evidence on what works, assessing cost-benefit, but also ensuring that the research results are presented in a clear accessible way – that underpinned the Teaching and Learning Toolkit the Trust developed for schools on what works best at improving the results of children from poorer backgrounds. The Toolkit has now been used by thousands of schools across the country, and underpins the work of the Education Endowment Foundation.

When we established the EEF in 2011 as lead foundation with Impetus our vision was that it was going to embrace the Sutton Trust’s principles and become a gigantic do tank. The aim was to improve the results of the poorest children in our most challenging schools. But it would also have the freedom to experiment, innovate and rigorously evaluate projects and scale up those that were cost effective.

Two years on I am pleased to say that this has become the reality. To date the EEF has awarded £24.4 million to 55 projects working with over 275,000 pupils in over 1,400 schools across England. It has commissioned over 40 randomised research trials in our schools – the gold standard for evaluations on what works. Over the coming years these studies will add greatly to our knowledge of what interventions are successful in the classroom.

But with research, you have to take the rough with the smooth. Not all the Sutton Trust’s research findings have been welcome. In 2005 the Trust jointly funded a five-year study with the Department for Business, Innovation, and Skills and the College Board into the US SAT aptitude test as a potential addition tool in the selection of candidates for universities.

In particular the National Foundation for Educational Research study aimed to find out whether the SAT test could identify highly able non-privileged students whose potential was not being reflected in A-levels because of their circumstances. After five years tracking the results of thousands sixth formers who then attending university, the study concluded that the SAT added little extra information to that provided by A-levels.

If the Government is true to its word on ‘evidence-based policy’ then it will have to face up to this reality. The research may not always confirm prior convictions or favoured policies, and almost always throws up some unexpected results. That’s why I think it is important the EEF and the Sutton Trust remain fiercely independent and make public all the evidence we produce. As the Government’s What Works evidence centre for education, these will be our guiding principles.

Why teachers can’t call themselves a profession

Lee Elliot Major on the need for an evidence-based approach in the classroom

I have often thought that commentators who want to criticise teachers should first pass the ‘teacher test’ to earn the right to do so. Having spent time in front of an inner-city classroom (with a teacher beside me) I can tell you it is one of the most challenging (and rewarding) experiences I have ever had. And that was just for one hour!

We don’t value our teachers anywhere near enough. Few of us really understand (or could cope with) the demands of their job. Just ask Sir Peter Lampl, a hard-nosed business leader, who once thought teachers had it easy. After 15 years at the helm of the Sutton Trust he now talks only of admiration for the inspirational educators of our children.

But teachers remain vulnerable to one well-founded attack. Can they call themselves a true modern-day profession? I’m afraid not. And one of the main reasons is this. Teachers have yet to embrace an evidence-based approach to their work: there is no accepted body of knowledge, based on robust research, to inform what they do (or don’t do); nor is there a culture of investigation to evaluate what works best in their particular school or classroom. The contrast with the modus operandi of medics could not be starker.

Below I have adapted a famous graph in education policy circles, first produced during New Labour’s early education reforms of the late 1990’s.

Knowledge poor

The graph describes the different phases of teaching during the last half century. It contrasts them in terms of the knowledge used to underpin the work of teachers, and the levels of autonomy they have enjoyed.

Before 1988 teachers were essentially practitioners free to pursue their own ways of working, with little reference to the body of research on what worked best. Then came the Big Bang of Baker’s Government reforms – the league tables, inspections and the national curriculum – that prescribed exactly what was expected in the classroom. A decade later under New Labour, another wave of top-down programmes emerged – the national numeracy and literacy strategies. These initiatives were based at least in part on the evidence of their impact.

The last phase of the graph highlights what has yet to be realised: the promised land of teachers as informed autonomous professionals, and no longer in need of Government direction.

The good news is that an accessible summary of education research on what works to raise attainment is now available. Last week saw the re-launch of the Sutton Trust-Education Endowment Foundation Toolkit. This is latest generation of the guide updating the evidence first released two years ago. It challenges many of the assumptions among teachers – revealing the limited impact from reducing class sizes and the current deployment of teaching assistants. This is one important step in the journey towards a teaching profession that embraces evidence. But as Ben Levin stresses, it is really only the start: everyone knows that exercise is good for them, but that doesn’t mean we all do it, does it?

I see at least three major challenges for the deep cultural reforms needed for teaching to evolve into the respected profession it should be.

The first is the glacial timescale for change. The improved education systems across the world required concerted leadership and efforts over several years. This is the “grind not the glamour” that the EEF’s chief executive, Kevan Collins, talks about.

The second is the danger of Whitehall’s heavy hand that can stifle rather than stimulate change. The long hard road to reform extends well beyond Whitehall fads (evidence based policy is currently back in vogue) and Parliamentary cycles.

No, this change will have to come from teachers themselves. It is striking that in all the reviews of the nations at the top of the global education rankings, the common watchwords are professionalism, professional development, evidence, and research.

Making the Pupil Premium go Further

Conor Ryan asks whether the Government is making the most of the pupil premium.

Most schools face standstill budgets, or real terms cuts.  But some of the school budget is growing, and will grow further. The pupil premium, worth £600 this year, rises to £900 next year and could reach £1200 per pupil by 2015. The challenge for schools is how to use that money where it will have most impact – and there is a case for incentivising effective practice.

The pupil premium was part of both the Liberal Democrat and Conservative manifestos. It had two important roles: firstly, to narrow the attainment gap between pupils on free school meals and their classmates, and secondly, to encourage successful schools to take more disadvantaged pupils.

Previous governments have provided extra resources for such pupils through extra funding to local authorities with high levels of poverty. Nevertheless, the Institute for Fiscal Studies has pointed out that pre-premium extra funding in the system attached to deprived pupils amounts to £2000 in primary schools and £3000 in secondary schools. And that has had variable impact.

But this is the first grant paid to schools for each disadvantaged pupil, regardless of where the school is located. The Government decided against ring-fencing the premium, relying instead on schools publishing details of spending on their websites and extra league table measures.

But there is mounting evidence that the premium is not being used as intended (nor is there any evidence that is changing admissions behaviour). Ofsted reported last month that often schools did not disaggregate the pupil premium from their main budget, and said that the funding was often used to maintain or enhance existing provision.

The National Foundation for Education Research, in a survey of 1700 teachers in 1200 English schools for the Sutton Trust earlier this year, showed that little of the pupil premium allocation for 2012-13 – a sum worth £1.25 billion in total – was likely to be spent on activities proven to be the best bets for boosting attainment. Many would spend it on staff rather than improving teaching.

But there is evidence of the most cost effective approaches to improving standards. Researchers at Durham University assessed 21 different interventions for both impact on attainment and relative cost and helped the Sutton Trust to create a toolkit which a growing number of schools are using priorities for the premium. The toolkit will be updated and expanded by our sister organisation, the Education Endowment Foundation, in 2013.

The Sutton Trust/EEF Teaching and Learning Toolkit shows that there are three strategies that schools can undertake with a high impact at relatively low cost. Using evidence from here and abroad, they calculate that each of these strategies can provide the equivalent of between six and nine months extra learning, at a cost of around £170 per pupil.

The single most effective strategy is improved pupil feedback, which should be an important part of a school’s drive to boost the quality of teaching and learning. Other low cost proven methods include teaching pupils to learn effectively and peer tutoring, where young people help each other to learn.

Of course, a growing number of schools are already turning to the Toolkit for ideas, and a new version will be online early next year. However, there is surely a case for the Government to help give proven ideas a further boost.

As the size of the pupil premium pot grows, why not use some of it to match fund schools’ spending on proven strategies? It is not as if the Government is opposed to ring-fencing any of the premium; Nick Clegg has used it to fund summer catch-up schools.

So, why not have a pot which schools where schools could double a proportion of their premium provided they spent it on a menu of proven activities? With a little extra incentive, ministers might find that the pupil premium went a lot further.

Debunking the Myths Around School Choice

Lee Elliot Major on the difference a school really makes to academic grades

“Stop looking around at everyone!” I  am back at school and being scolded again. But not by a teacher. It is my better-half who is telling me off, as I crank my neck around to see who else is present in the packed school hall. We are sitting among rows of similarly anxious parents and fidgety children waiting for the head-teacher to deliver her ‘pitch’. Then, ten minutes later, we are swept off by a Year 9 pupil on a dizzying tour of colourful corridors, computer labs, and crowded classrooms.

Like hundreds of thousands of parents up and down the country, the start of this academic season signals the search for the right secondary school for one of our children.

In North London, this is a particularly tortuous time. Guilt-ridden liberal-minded middle-class types are at pains to justify their decision to abandon the state system for private schools elsewhere. And those who have committed to the local Comp have been transformed into evangelical zealots. There is no end in sight for England’s ancient education divide.

But what is shocking to me is that otherwise highly intelligent, sensible, well-balanced parents appear to be making school decisions based on hearsay, second-hand rumours, and, probably most damaging of all, some basic education myths and misunderstandings.

Probably the single biggest myth is that a child’s academic grades are mostly determined by the school they attend. If school A has 100% of children with 5 GCSEs including English and maths, and school B has only 50% meeting this benchmark, then school A must be twice as good as school B, right?

Well, actually no. One of the few unassailable findings of education research over the last five decades is the ’80-20 rule’: roughly 80% of the variation in children’s school results is due to individual and family characteristics outside the school gates, with the remaining 20% due to what actually happens during school.

You can do the maths in terms of final GCSE grades, and it produces a surprising result. The impact of an average pupil going to the very best performing school in the country rather than the worst performing school would be an improvement in his or her results by one GCSE grade, say from a C and to a B for each GCSE taken.

There are important rejoinders to this finding. Some schools can and do have a bigger impact than others – as shown by markedly different academic results despite very similar intakes of children. Parents would do well to look at the wealth of Government data now available for individual schools, showing progress for different groups of pupils. Even a cursory look will prompt some challenging questions for teachers. And inspection reports can provide useful context alongside these figures.

It is also true that in an increasingly competitive education race, one extra grade at GCSE can make the critical difference. It can determine, for example, whether you get into a selective sixth form.

But this doesn’t detract from the overwhelming message that if you are a child from a secure and supportive background, then you will do well academically at most schools. That puts the whole question of school choice into a completely different perspective.

In fact probably the best question parents can ask is how a school is addressing the variation in teaching quality across a school. There is no public data available on this. But education research tells us that the biggest differences in results is within schools not between them. We also know that teacher quality is the key driver – not class size, not school type, not uniform, certainly not school buildings.

This is one of the major messages of the Sutton Trust-EEF teaching and learning toolkit, a guide to the most cost-effective approaches for improving attainment in schools. Originally commissioned by the Sutton Trust, the toolkit is now being developed by the Education Endowment Foundation – and should be compulsory reading for every parent.

There are of course many things apart from academic results to consider when thinking about a school. But if you simply want to know what the school’s average GCSE grades will be in 5 years time, then just look around you at who else is present during the school’s open day. The answer is staring you in the face.