All posts by Samuel Moore

Open Research in the Humanities: Research Evaluation 

This is the sixth and final of a series of blog posts, presenting the reflections of the Working Group on Open Research in the Humanities.  The working group aimed to reframe open research in a way that was more meaningful to humanities disciplines, and their work will inform the University of Cambridge approach to open research.   

This post discusses opportunities and challenges for research evaluation in the arts and humanities. 

The direction of travel in the Open Research discussion is away from any straightforward use of metrics in research evaluation. This is hugely in favour of the A&H. 

Opportunities 

The A&H have never used metrics in the same way as their STEM colleagues. This is partly because of the slower speed of publishing and the in-depth editorial process (18-24 months from submission to publication might be considered standard), and partly because ‘citation indices’ are less relevant when one’s contribution is to be part of a broad, ongoing cultural conversation rather than to generate data from scratch (see above, on CORE data). So the diversification of research evaluation enshrined in DORA (https://sfdora.org), and the questioning of the uncritical use of metrics and altmetrics by administrators, grant funders and promotion committees, is a positive development. It allows for a general move away from established academic platforms and formats, as discussed above.  

Support required 

Some pressing questions about research evaluation remain, which might account for some perceived hostility from some quarters towards a move away from established academic platforms. Who is doing the work of reading and assessing these multiple new formats? How do we evaluate success? Is success measured in terms of ‘reach’ – number of Twitter followers or blog readers etc? This would take us down the route of clickbait and skew towards already-popular, English-language material; this is a particular danger with processes designed to evaluate web traffic.  

What guidance is available for established academics looking to credit other colleagues for their social media contributions, in particular? An anxiety often expressed is that ‘there is a lot of rubbish on the internet’. How do we sift through? In general, research evaluation takes time and effort, and there is a sense that this work needs to be properly measured and quantified. For example, if an evaluator spends 20 minutes per CV on 60 CVs, that is 20 hours of work before one even gets into reading and evaluating actual outputs. In the context of a busy teaching term, such additional labour is barely possible and contributes to a general sense of stress within the profession. Looking for a kind of shorthand to facilitate swift and accurate evaluation of a wide range of (possibly unfamiliar) formats is therefore the pragmatic approach.  

The discussion of narrative CVs in the DORA context implies an amalgamation of the traditional CV and the cover letter. In our institution as no doubt in others too, it would be useful to have some HR guidance here on what appointment panels should ask for (e.g. no cover letter, but a paragraph each on a candidate’s three main research achievements?).  

There was a feeling in the working group that Cambridge is perhaps behind other universities who make ‘open research’ a category for assessment in itself, and who guide their employment panels and candidates accordingly. Indeed, Cambridge’s traditional division of the criteria for promotion into the discrete categories of ‘research’, ‘teaching’ and ‘general contribution’ seems actively to work against the whole idea of ‘open research’. It suggests unhelpfully that ‘service to the community’, ‘teaching’ and ‘research’ do not overlap. This division seems to have survived the recent overhaul of the academic promotions exercise.  

Thoughts on the new White House OSTP open access memo

Dr. Samuel A. Moore, Scholarly Communication Specialist, Cambridge University Libraries

In the USA last Thursday, the White House Office of Science and Technology Policy announced its decision to mandate public access to all federally funded research articles and data. From 2026, the permitted embargo period of one year for funded publications will be removed and all publications arising from federal funding will have to be immediately accessible through a repository. Although more details are to be announced, my colleague Niamh Tumelty, the OSC’s Head of Open Research Services, shared a helpful summary of the policy and some initial reaction here. I want to offer my own personal assessment of what the new policy might mean from the perspective of open access to research articles, something we are working hard to promote and support throughout the university.

To be sure, the new OSTP memo is big news: the US produces a huge amount of research that will now be made immediately available without payment to the world at large. Following in the footsteps of Plan S in Europe, the open access policy landscape is rapidly evolving away from embargo periods and towards immediate access to research across all disciplines. Publishing industry consultants Clarke & Esposito have even argued that this intervention will make the subscription journal all the more unviable, eventually leading to its demise.

Indeed, responses from the publishing industry have been mixed. The STM Association, for example, offer a muted one-paragraph response claiming tepid support for the memo, while organisations such as the AAP were more vocally against what they see as a lack of ‘formal, meaningful consultation or public input’ on the memo, despite the fact that many more details are still to be announced (presumably, following consultation). A similar sense of frustration was displayed by some of the authors of the industry-supported Scholarly Kitchen blog. It’s fair to say that the publishing industry itself – at least the part of it that makes money from journal subscriptions – has not welcomed the new memo with open arms.

Understandably, funders and advocacy organisations have welcomed the news. Johan Rooryck from Coalition S called the memo a ‘game changer for scholarly publishing’, while the Open Research Funders Group ‘applauds bold OSTP action’ in its response. Open access advocates SPARC described the memo as a ‘historic win’ for open access and a ‘giant step towards realizing our collective goal of ensuring that sharing knowledge is a human right – for everyone’. Certainly, for those arguing in favour of greater public access to research, the memo will indeed result in just this. But I still have my reservations.

My PhD thesis analysed and assessed the creation and implementation of open access policy in the UK. As Cambridge researchers no doubt know, the open access policy landscape is composed of a number of mandates, with varying degrees of complexity, and affects the vast majority of UK researchers in one way or another. This is for better and for worse: there is an increase in bureaucracy associated with open access policy (particularly through repositories), even though it results in greater access to research. However, when you remove this bureaucracy through more seamless approaches to OA like transformative agreements, there is a risk of consolidating the power of large commercial publishers who dominate this space and make obscene profits (a fear also shared by Jeff Pooley in his write-up of the policy). There is therefore a delicate balance to be struck between simply throwing money at market-based solutions and requiring researchers and librarians to take on more of the burden of compliance.

The problem with indiscriminate policy mandates for public access to research, such as the OSTP’s memo, is that they shore up the idea that publishing has to be provided by a private industry that is not especially accountable to research communities or the university more broadly. This is precisely because these policies are indiscriminate and therefore apply to everyone equally, which for academic publishing means benefitting those already in a good position to profit. Larger commercial publishers have worked out better than anyone else how to monetise open access through a range of different business models. As long as researchers need to continue publishing with the bigger publishers, which they do for career reasons, these publishers will always be in a better position to benefit from open access policies. It is hard to imagine how the individual funding bodies could implement the OSTP memo in a way that does foreground a more bibliodiverse publishing system at the expense of commercialism (not least because this goal does not appear to be the target of the memo).  

I do not mean to overplay the pessimism here: it is great that we are heading for a world of much more open access research. The point now is to couple this policy with funding and support to continue building the capacity of an ethical and accountable publishing ecosystem, all while trying to embed these ethical alternatives within the mainstream. This kind of culture change cannot be achieved by mandates like the OSTP is proposing, but it can be achieved by the harder work of raising awareness of alternatives and highlighting the downsides of current approaches to publishing. It is also important to reveal the ways in which research cultures shape how researchers decide to publish their work – often at the expense of experimentation and openness – and how they can be changed for the better.

So I am interested to see how the memo is implemented in practice, especially how it is funded and the conditions set on immediate access to research. I am also keen to see what role, if any, rights retention plays in the implementation and how US libraries decide to support the policy and the changing environment more broadly. Ultimately, however, the move to a more scholar-led and scholar-governed ecosystem will not occur on an open/closed binary, nor on a top-down/bottom-up one, and so we must find a range of ways to support new cultures of knowledge production and dissemination in the university and beyond.

Image taken from Public Domain Pictures

Panel summary: open access monographs without author payments?

The Open Research at Cambridge conference took place between 22–26 November 2021. In a series of talks, panel discussions and interactive Q&A sessions, researchers, publishers, and other stakeholders explored how Cambridge can make the most of the opportunities offered by open research. This blog is part of a series summarising each event. 

As part of the 2021 Open Research Conference at Cambridge, we hosted a panel discussion on the future of open access monographs, specifically those that do not require author payment in the form of book processing charges. This is especially timely given the fact that UK Research and Innovation recently announced a books component to its open access policy. In the humanities, where funding is limited, book processing charges have the potential to make open access a preserve of only those that can afford to pay, potentially excluding junior scholars, unfunded researchers and colleagues from universities outside the Global North. This panel therefore explored the alternatives to author payments that exist and the ways in which the research community can prevent processing charges from becoming the standard model for open access book publishing.

The panel was moderated by Samuel Moore, Scholarly Communication Specialist at the Office of Scholarly Communication, and featured three expert speakers from publishing and policymaking. Kicking off was Rachel Bruce, Head of Research at UK Research and Innovation, to discuss their open access books policy that was recently announced. We learned more of the details of the policy but also how much of the detail relating to funding is yet to be announced. Following Rachel’s presentation were talks by Ben Denne of Cambridge University Press and Rupert Gatti of Open Book Publishers. Ben described the Flip it Open model devised by Cambridge University Press that makes a book openly accessible when it reaches a certain revenue threshold. Rupert described the overall approach of Open Book Publishers to make books available without the need for author payment, through a combination of print sales, grant income and other sources.

The ensuing discussion covered important topics such as the sustainability of such approaches, the technologies that underpin them and the importance of green open access for open access book publishing. We learned that funding needs to be equitably distributed to enable a diverse ecosystem of OA presses and that there is no one-sized model for open access book publishing. The work for this is at once the responsibility of policymakers, publishers, librarians and researchers alike.

Additional resources

Open Book Publishers

UKRI Open Access Policy

University of Cambridge Monographs guidance

Cambridge University Press ‘Flip it Open’ pilot