neurotaylor

Wearily kissing the rod: UK academics and the REF

(a long post, this one, about who gets research money)

The Research Excellence Framework (REF) is the UK government’s way of assigning money for academic research.

It’ll happen in 2014. Work to prepare for it has been ongoing since its predecessor, the Research Assessment Exercise, finished, if not before.

As Jenny Rohn has recently commented in the Guardian,and many others have also said, the REF is a real problem for UK science, and for academia in general. To solve that problem, we need to understand why the REF is happening.

The core issue, as the government sees it, is that limited resources and the pressure to provide value for money (right now) create a demand that money should only be spent on good research. Unfortunately, it can take decades to work out which research is good.

There’s also considerable debate about what good research is. Academics tend to think of good research as being scholarly: rigorous in its methods, intellectually coherent and persuasive, novel and insightful (it adds new knowledge), enriching (it adds new understanding, e.g. by relating previously unrelated phenomena), and fruitful (it generates lots of new ideas). Governments and funders seem to think of good research as being efficient at generating lots of future research jobs, cash for the economy, or positive publicity for UK PLC. But scholarship can cost money and take time without producing immediate economic rewards, so it appears inefficient over the short term of a government (5 years) or a REF assessment cycle (4 years).

What to do? Well, you could try short-circuiting the judgement of posterity by asking current academics what they think of other people’s research. Brilliant! That ticks all the boxes of consultation, transparency, stakeholder participation, and so on.

Unfortunately, it also has its problems. For one, there is as yet no firm evidence that having an assessment system like the REF improves research quality. Senior academics like Stefan Collini have argued that it will make things worse, not better. But for all that, it’s going ahead with surprisingly little protest from academia.

Why? Here are some reasons that may be relevant.

1) It’s conservative. The REF emasculates one major source of resistance, the ideal of academic independence, because the government can turn round to academics and say, ‘Hey, you’re the people making the judgements, all we’re doing is divvying up the funding at the end’. Actually, it’s not so much the academics as the university managers who make the judgements, because they set the criteria for who’s ‘REFable’ among their staff. This intensifies conservative tendencies, because fashionable topics are rewarded more; yet academia needs more new ideas, not fewer. Because universities do much of the rating in advance, by choosing whose work to submit to the REF, that conservatism sets in early. Because it is more senior staff who do much of the rating, that adds to the risk that new ideas will be squeezed out.

Making institutions choose whom they submit is the biggest flaw in the system, in my view. Academics need to be judged by their peers — other academics who are expert in their field. Asking a psychologist to rate the work of a neuroscientist in the same university department might make sense if the psychologist works on something brain-related, but if they don’t? If the psychologist’s field is the closest in that department to what the neuroscientist is doing, chances are they’ll get to decide their REFability, whether or not they speak the same language, think about science in the same way, or understand the research. Let’s hope the psychologist isn’t an undeclared cat lover who rates the neuroscientist down because they’re doing animal research. At least in a grant application you’ve some chance that assessors will be experts in your field!

2) It’s bad for diversity. The REF structure allows government to externalise much of the cost. Only those submitted as REFable will then have their work assessed by an expert panel. The considerable downside is discrimination against minority topics, which are less likely to be assessed by expert colleagues in the same institution, since there may not be any. Good scholarship should be rewarded whatever the topic, but in practice that doesn’t happen. Instead, people look at a piece of work, think, ‘this is a marginal subject’, or ‘I don’t like this person’s attitude’, and downgrade it in REF terms, whatever its intellectual quality. Meanwhile, there’s plenty of fashionable nonsense earning plaudits, even though it is less rigorous, less intellectually coherent, perhaps less fruitful (and often not even as well-written). Academics aren’t immune to the human herd instinct, and the REF gives it free rein. Work published in top-ranking journals and by top-ranking international publishers can thus be ranked so low by colleagues that it may never get as far as a REF expert panel. The solution is to have all academics submitted, not just those selected by the institution.

3) It’s unclear. The lack of clarity in the REF has been a gift to those running it and a source of considerable stress to those being processed. Many academics, particularly those at junior and middle levels, are now so demoralised that they dare not speak out. There are rumours that those not ‘returned’ (submitted to the REF) will be taken off research onto teaching-only contracts if they are not REF-returned. There is a strong feeling that any adverse comment could mark you as a troublemaker in the eyes of those with power to ruin your career. University managers and government ministers could state, for the record, that they have no such plans, and that academics are allowed, and expected, to criticise how their universities are run. That is not the message that is currently reaching staff. They don’t know what to think, because the criteria are so vague; and uncertainty breeds fear — which the leadership has largely left to fester. Of course it has. Demoralised academics are far less trouble.

4) It’s burdensome. Staff can also be demoralised by adding to their workload, and the REF has done this too.  Those called to assess submissions (which may be papers, or books, four pieces per person) must do the work alongside their ordinary jobs. Universities are even hiring people to help them with their REF submissions. They’re also wasting loads of staff time on REF bureaucracy, further stressing already overworked academics — who fill in the damn forms in what should be their research time, and then risk being punished for not doing enough research!

5) It’s bad for morale. Staff can be demoralised even more by increasing the emphasis on competition. Academia is in part competitive, but the traditional model was inter-institutional competition (think the Oxford-Cambridge Boat Race!). Institutions which looked after their own fostered a collegiate culture which allowed for the development of a ‘critical mass’ of intellectual cooperation. Academics who felt secure enough to share their early ideas with others were able to benefit from their comments, to cross-fertilize ideas, to generate new hypotheses and new collaborations. If every person is out for his or her own career, and forced to compete with the people he also has to work with, that causes intense psychological discomfort. It also reduces creativity. Years after science discovered that competition isn’t what life’s all about after all, university management is still stuck in the 1980s. The REF encourages game-playing. Universities are now hiring science writers to help craft their impact case studies. That money could be going on better teaching provision.

6) It’s short-sighted. Finally, there’s the obvious point that excellence doesn’t spring from a void. Like a sought-after orchid, it needs an entire ecosystem to flourish. Keep picking the flowers, moreover, and the number of orchids will drop. The REF’s focus on excellence, so appealing in principle, is not so clever in practice, because it risks creating an academic wasteland. Healthy ecosystems, of course, have weeds and other apparently useless plants. Weeding out useless academics is a popular goal, but using the REF to do this is like weeding a garden with a JCB. Weeding by hand, however, would demand a lot of time, skill and effort from university management.

In all of this, the REF reminds me, in a mild way, of the techniques of ‘thought reform’ used by the Chinese Maoists to subjugate political dissenters. They set people against each other, constantly reviewing and criticising them, and forcing them to criticise each other. Hard, mind-numbingly tedious work also helped wear the dissidents down, as did the fear and uncertainty in which they were kept. The thought reformers knew that unspecified threats of punishment can be more effective than actually listing consequences, and they were careful to leave their criteria for good and bad behaviour vague and flexible. Interminable discussions of ideology were coupled with frustrating lack of clarity on the details; one might therefore sin without understanding how. This gave the impression that judgement of who was ideologically correct was essentially arbitrary.

The result was obedience, with or without commitment. People could be brought to kiss the rod that beat them, to praise the ideals that oppressed them, or at least to keep sullenly, fearfully quiet.

Of course, analogy is not identity. Thought reform was conducted in prison camps, and used torture. Academics are free to leave universities, if they don’t mind going on the dole (jobs are scarce, and speaking out against university management is unlikely to make them less scarce). As for hiking workloads, imposing arbitrary bureaucratic burdens which shrink research time, taking away the sense of having any control over teaching or admin, and making staff feel that they are at the bottom of the status heap, these techniques may do psychological harm, but they’re not torture.

But nor are they a sensible way to run higher education. The gains in efficiency are short-term, and largely illusory. They seem desirable because the government’s research funds are not handed out by the same people who are paying even for the teaching budget, let alone for the sick leave of overstretched academics. Nor are today’s votes significantly affected by future reductions in academic creativity.

The REF is pernicious and deeply flawed. Many people have pointed out the problems — yet it’s going ahead. When that happens, it’s time to start asking cui bono? Is the continued existence of the REF just down to inertia in the system, or might those flaws be there for good political reasons?

So who does stand to benefit from the REF? Academics? The most senior ones, perhaps, in the most highly-regarded subjects. But not the rest.

The advantages for government and management, however, are obvious. They can claim to be pursuing efficiency and excellence and put the blame for any future failures in research on universities. They also lessen the chances of a revolt among academic staff by the time-tried tactics of divide-and-rule, distraction, and exhaustion. And they save money by making those same academics do most of the work.

Seen from this angle, the REF starts to look like rather a good thing. Could that be why we still have it?

Related articles

• Must all postgrad research have ‘impact’? (guardian.co.uk)
• Dodgy dealings in UK higher education | Jenny Rohn (guardian.co.uk)
• How to defend universities | Paul Seabright (Times Literary Supplement)