Open Research in the Humanities: Research Evaluation 

This is the sixth and final of a series of blog posts, presenting the reflections of the Working Group on Open Research in the Humanities.  The working group aimed to reframe open research in a way that was more meaningful to humanities disciplines, and their work will inform the University of Cambridge approach to open research.   

This post discusses opportunities and challenges for research evaluation in the arts and humanities. 

The direction of travel in the Open Research discussion is away from any straightforward use of metrics in research evaluation. This is hugely in favour of the A&H. 

Opportunities 

The A&H have never used metrics in the same way as their STEM colleagues. This is partly because of the slower speed of publishing and the in-depth editorial process (18-24 months from submission to publication might be considered standard), and partly because ‘citation indices’ are less relevant when one’s contribution is to be part of a broad, ongoing cultural conversation rather than to generate data from scratch (see above, on CORE data). So the diversification of research evaluation enshrined in DORA (https://sfdora.org), and the questioning of the uncritical use of metrics and altmetrics by administrators, grant funders and promotion committees, is a positive development. It allows for a general move away from established academic platforms and formats, as discussed above.  

Support required 

Some pressing questions about research evaluation remain, which might account for some perceived hostility from some quarters towards a move away from established academic platforms. Who is doing the work of reading and assessing these multiple new formats? How do we evaluate success? Is success measured in terms of ‘reach’ – number of Twitter followers or blog readers etc? This would take us down the route of clickbait and skew towards already-popular, English-language material; this is a particular danger with processes designed to evaluate web traffic.  

What guidance is available for established academics looking to credit other colleagues for their social media contributions, in particular? An anxiety often expressed is that ‘there is a lot of rubbish on the internet’. How do we sift through? In general, research evaluation takes time and effort, and there is a sense that this work needs to be properly measured and quantified. For example, if an evaluator spends 20 minutes per CV on 60 CVs, that is 20 hours of work before one even gets into reading and evaluating actual outputs. In the context of a busy teaching term, such additional labour is barely possible and contributes to a general sense of stress within the profession. Looking for a kind of shorthand to facilitate swift and accurate evaluation of a wide range of (possibly unfamiliar) formats is therefore the pragmatic approach.  

The discussion of narrative CVs in the DORA context implies an amalgamation of the traditional CV and the cover letter. In our institution as no doubt in others too, it would be useful to have some HR guidance here on what appointment panels should ask for (e.g. no cover letter, but a paragraph each on a candidate’s three main research achievements?).  

There was a feeling in the working group that Cambridge is perhaps behind other universities who make ‘open research’ a category for assessment in itself, and who guide their employment panels and candidates accordingly. Indeed, Cambridge’s traditional division of the criteria for promotion into the discrete categories of ‘research’, ‘teaching’ and ‘general contribution’ seems actively to work against the whole idea of ‘open research’. It suggests unhelpfully that ‘service to the community’, ‘teaching’ and ‘research’ do not overlap. This division seems to have survived the recent overhaul of the academic promotions exercise.  

A new institutional open access fund for the University of Cambridge

Open Access is a powerful tool that enables researchers to share their research and maximise the impact of their work. However, the reality is that gold open access is a business model that is based on paying to publish, and it’s a business model that is primarily supported by research funders. What that means in practice is that gold open access often comes with a price tag that effectively excludes unfunded researchers.

The University of Cambridge has established a new institutional open access fund to provide financial support for unfunded researchers across the collegiate University. Researchers who do not have access to grant funds with which to pay the open access fees will be able to use the fund to pay the open access fees for their research or review papers in fully open access journals.  

Professor Anne Ferguson-Smith, Pro-Vice-Chancellor for Research explains that:

“This is significant step in ensuring that all University of Cambridge researchers can opt for gold open access publishing. We are proud to establish this fund that will be especially beneficial to early career researchers as well as other researchers in the collegiate University who are not eligible for the open access funds that are provided by grant funders. Significant inequalities remain in the global scholarly publishing system, however, so we continue our commitment to support different open access solutions that are available to any researchers, both within and outside the University”.

The new fund is one of the many ways that the University helps researchers make their research open access and complements our many Read & Publish deals, the Rights Retention Pilot and the facilitation of green self-archiving in the University’s institutional repository, Apollo. 

Who can use the new institutional open access fund?

Researchers who have a strong connection (typically research staff and students) with the University of Cambridge (including the Colleges) who have no way of paying for open access fees in fully open access journals.

What will the new institutional open access fund cover?

The fund can be used to pay open access fees for full research articles or non-narrative review papers in fully open access journals, provided there is no other way of paying the fee (for example, where there is no Read & Publish deal available to any of the authors, or where none of the author team has access to any funding).

How will the new institutional open access fund work?

The Open Access team can provide in principle funding decisions but cannot guarantee payment until a paper has been accepted for publication. Researchers are encouraged to seek an in-principle decision before incurring any fees by emailing the Open Access Team.

Where can I find more information about the new institutional open access fund?

There is more information on the Open Access website about the institutional fund, and the Open Access Team is available to answer any queries.

What about the bigger issue?

We are very conscious about the wider challenges with author-pays models of open access, for example for unaffiliated researchers and those in institutions without access to funding of this sort, and especially of the global equity issues that arise. We held several strategic workshops looking at these issues earlier this year and will continue to work towards finding a more equitable future for open access scholarly publishing.

Thoughts on the new White House OSTP open access memo

Dr. Samuel A. Moore, Scholarly Communication Specialist, Cambridge University Libraries

In the USA last Thursday, the White House Office of Science and Technology Policy announced its decision to mandate public access to all federally funded research articles and data. From 2026, the permitted embargo period of one year for funded publications will be removed and all publications arising from federal funding will have to be immediately accessible through a repository. Although more details are to be announced, my colleague Niamh Tumelty, the OSC’s Head of Open Research Services, shared a helpful summary of the policy and some initial reaction here. I want to offer my own personal assessment of what the new policy might mean from the perspective of open access to research articles, something we are working hard to promote and support throughout the university.

To be sure, the new OSTP memo is big news: the US produces a huge amount of research that will now be made immediately available without payment to the world at large. Following in the footsteps of Plan S in Europe, the open access policy landscape is rapidly evolving away from embargo periods and towards immediate access to research across all disciplines. Publishing industry consultants Clarke & Esposito have even argued that this intervention will make the subscription journal all the more unviable, eventually leading to its demise.

Indeed, responses from the publishing industry have been mixed. The STM Association, for example, offer a muted one-paragraph response claiming tepid support for the memo, while organisations such as the AAP were more vocally against what they see as a lack of ‘formal, meaningful consultation or public input’ on the memo, despite the fact that many more details are still to be announced (presumably, following consultation). A similar sense of frustration was displayed by some of the authors of the industry-supported Scholarly Kitchen blog. It’s fair to say that the publishing industry itself – at least the part of it that makes money from journal subscriptions – has not welcomed the new memo with open arms.

Understandably, funders and advocacy organisations have welcomed the news. Johan Rooryck from Coalition S called the memo a ‘game changer for scholarly publishing’, while the Open Research Funders Group ‘applauds bold OSTP action’ in its response. Open access advocates SPARC described the memo as a ‘historic win’ for open access and a ‘giant step towards realizing our collective goal of ensuring that sharing knowledge is a human right – for everyone’. Certainly, for those arguing in favour of greater public access to research, the memo will indeed result in just this. But I still have my reservations.

My PhD thesis analysed and assessed the creation and implementation of open access policy in the UK. As Cambridge researchers no doubt know, the open access policy landscape is composed of a number of mandates, with varying degrees of complexity, and affects the vast majority of UK researchers in one way or another. This is for better and for worse: there is an increase in bureaucracy associated with open access policy (particularly through repositories), even though it results in greater access to research. However, when you remove this bureaucracy through more seamless approaches to OA like transformative agreements, there is a risk of consolidating the power of large commercial publishers who dominate this space and make obscene profits (a fear also shared by Jeff Pooley in his write-up of the policy). There is therefore a delicate balance to be struck between simply throwing money at market-based solutions and requiring researchers and librarians to take on more of the burden of compliance.

The problem with indiscriminate policy mandates for public access to research, such as the OSTP’s memo, is that they shore up the idea that publishing has to be provided by a private industry that is not especially accountable to research communities or the university more broadly. This is precisely because these policies are indiscriminate and therefore apply to everyone equally, which for academic publishing means benefitting those already in a good position to profit. Larger commercial publishers have worked out better than anyone else how to monetise open access through a range of different business models. As long as researchers need to continue publishing with the bigger publishers, which they do for career reasons, these publishers will always be in a better position to benefit from open access policies. It is hard to imagine how the individual funding bodies could implement the OSTP memo in a way that does foreground a more bibliodiverse publishing system at the expense of commercialism (not least because this goal does not appear to be the target of the memo).  

I do not mean to overplay the pessimism here: it is great that we are heading for a world of much more open access research. The point now is to couple this policy with funding and support to continue building the capacity of an ethical and accountable publishing ecosystem, all while trying to embed these ethical alternatives within the mainstream. This kind of culture change cannot be achieved by mandates like the OSTP is proposing, but it can be achieved by the harder work of raising awareness of alternatives and highlighting the downsides of current approaches to publishing. It is also important to reveal the ways in which research cultures shape how researchers decide to publish their work – often at the expense of experimentation and openness – and how they can be changed for the better.

So I am interested to see how the memo is implemented in practice, especially how it is funded and the conditions set on immediate access to research. I am also keen to see what role, if any, rights retention plays in the implementation and how US libraries decide to support the policy and the changing environment more broadly. Ultimately, however, the move to a more scholar-led and scholar-governed ecosystem will not occur on an open/closed binary, nor on a top-down/bottom-up one, and so we must find a range of ways to support new cultures of knowledge production and dissemination in the university and beyond.

Image taken from Public Domain Pictures