For the past few days my little corner of the Internet has been filled with reflections on the just released Stern report on the UK’s Research Excellence Framework (REF). To keep things simple for non-UK readers, REF is an assessment exercise held every five to seven years that the UK government uses to allocate block grants to universities. Not surprisingly, then, universities are always looking for ways to score high on REF and thereby maximise funding. [Actually, I think REF’s influence is disproportionate to the funding that it allocates, but that’s another issue.]
I have to admit that my first reaction when I saw Lord Stern’s recommendations were that if these had been in place four years ago I might not have gotten my current job! Two of the points made by the department to the university to justify my hire were that I would bring publications with me (for the 2014 REF) and that I would provide a means for tying research to an Impact Case Study (for the anticipated 2021 REF). Stern recommends that the former be eliminated and that the latter be deemphasised.
Much of the discussion about the report has centred on proposed changes regarding who gets included in a REF submission, how many publications they can (or must) submit, and what can count as a submitted publication (see, for instance, commentary by by Stuart Elden and Andrew McRae, as well as a general list of responses and resources [thank you Stuart for this link]). However, I find the most interesting part of the report Lord Stern’s recommendation to explicitly broaden the criteria for what can count as ‘Impact’.
To be blunt, I think impact is a good thing. In fact, I imagine that all academics hope that they will make a difference beyond the world of their academic peers. Certainly I would like to use the space that my job gives me to do relatively ‘pure’ research wherein there are no pre-defined ‘deliverables’ to make a difference to a wider, more diverse audience than might be the case if I held a more conventional, directed job. (I’ll resist the urge to insert a clichéd reference to Marx’s 11th Thesis on Feuerbach here, but, well, you know what I mean.)
The devil, of course, is in the details, though. When I moved from the US to the UK I had assumed that ‘Impact’ was roughly equivalent to the ‘Broader Impacts’ that one claims one will have when one writes a US National Science Foundation grant proposal. The NSF’s ‘Broader Impacts’ cover a range of ways in which research can make a difference beyond the specific research objectives being addressed. Often in a ‘Broader Impacts’ statement one writes that one will integrate knowledge gained during a research grant into a teaching curriculum, or that one will train PhD students or teach new techniques to postdocs, or that one will develop a public-facing version of one’s research output so that findings can be shared with a wider audience and thereby increase public knowledge. None of this, I soon found out, counted for much in the UK, where ‘Impact’ is defined as ‘an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.’ This is a fascinating (and troubling) definition for what it doesn’t include. In particular, increasing knowledge, awareness, or understanding outside academia is notably missing from this definition. (These ‘ancillaries’ to research currently fall under the different term ‘Engagement’.) Also, by lumping all university-directed knowledge within the term ‘academia’, the definition ensures that any spillover that research may have on teaching will fall outside the ‘impact’ category (since they occur within ‘academia’). This is in particularly sharp contrast with the US where impacts that directly benefit education are specifically valued; indeed, the NSF has an entire grant scheme that encourages early career researchers to develop these synergies, presumably because it is aware that otherwise promising early career reseacrhers will forsake teaching innovations as they succumb to university grant and publication pressures.
The Stern Report goes a long way to rectifying some of these problems with the UK ‘Impact’ agenda. It recommends that engagement (public outreach) and teaching-led innovations tied to research should count as impact, and from my perspective that’s a very good thing.
At the same time, though, these reforms just begin to address underlying problems. If the UK funding councils were really serious about breaking down the barriers between research and teaching in higher education, then they’d facilitate linking PhD studentships with research grants. At present, there is a strong incentive to hire postdocs instead of PhD students; indeed, in most cases it’s not even permitted to use research council research funds to support PhD students. While I can see where bringing on postdocs might maximise short-term efficiency in generating research findings, funding PhD students likely would have a longer-lasting effect on linking research with a university’s educational mission. And while the proposed inclusion of public engagement within ‘Impact’ is a welcome change, it only begins to address how the gap between university education and an informed, engaged general public is still maintained by a lack of openness to non-traditional (part-time, older, child-caring, etc.) students.
We probably shouldn’t look for too much from the REF. It’s likely not the most effective or appropriate mechanism for addressing all of the distortions and inequities of British education (many of which, of course, reach deep into social structures and cultural history). However, as a funding mechanism it certainly carries some weight. And to this end the reforms proposed by Lord Stern, at least with respect to redefining ‘Impact’, represent a step in the right direction.