Category Archives: Humanities

Open Research in the Humanities: Research Evaluation 

This is the sixth and final of a series of blog posts, presenting the reflections of the Working Group on Open Research in the Humanities.  The working group aimed to reframe open research in a way that was more meaningful to humanities disciplines, and their work will inform the University of Cambridge approach to open research.   

This post discusses opportunities and challenges for research evaluation in the arts and humanities. 

The direction of travel in the Open Research discussion is away from any straightforward use of metrics in research evaluation. This is hugely in favour of the A&H. 

Opportunities 

The A&H have never used metrics in the same way as their STEM colleagues. This is partly because of the slower speed of publishing and the in-depth editorial process (18-24 months from submission to publication might be considered standard), and partly because ‘citation indices’ are less relevant when one’s contribution is to be part of a broad, ongoing cultural conversation rather than to generate data from scratch (see above, on CORE data). So the diversification of research evaluation enshrined in DORA (https://sfdora.org), and the questioning of the uncritical use of metrics and altmetrics by administrators, grant funders and promotion committees, is a positive development. It allows for a general move away from established academic platforms and formats, as discussed above.  

Support required 

Some pressing questions about research evaluation remain, which might account for some perceived hostility from some quarters towards a move away from established academic platforms. Who is doing the work of reading and assessing these multiple new formats? How do we evaluate success? Is success measured in terms of ‘reach’ – number of Twitter followers or blog readers etc? This would take us down the route of clickbait and skew towards already-popular, English-language material; this is a particular danger with processes designed to evaluate web traffic.  

What guidance is available for established academics looking to credit other colleagues for their social media contributions, in particular? An anxiety often expressed is that ‘there is a lot of rubbish on the internet’. How do we sift through? In general, research evaluation takes time and effort, and there is a sense that this work needs to be properly measured and quantified. For example, if an evaluator spends 20 minutes per CV on 60 CVs, that is 20 hours of work before one even gets into reading and evaluating actual outputs. In the context of a busy teaching term, such additional labour is barely possible and contributes to a general sense of stress within the profession. Looking for a kind of shorthand to facilitate swift and accurate evaluation of a wide range of (possibly unfamiliar) formats is therefore the pragmatic approach.  

The discussion of narrative CVs in the DORA context implies an amalgamation of the traditional CV and the cover letter. In our institution as no doubt in others too, it would be useful to have some HR guidance here on what appointment panels should ask for (e.g. no cover letter, but a paragraph each on a candidate’s three main research achievements?).  

There was a feeling in the working group that Cambridge is perhaps behind other universities who make ‘open research’ a category for assessment in itself, and who guide their employment panels and candidates accordingly. Indeed, Cambridge’s traditional division of the criteria for promotion into the discrete categories of ‘research’, ‘teaching’ and ‘general contribution’ seems actively to work against the whole idea of ‘open research’. It suggests unhelpfully that ‘service to the community’, ‘teaching’ and ‘research’ do not overlap. This division seems to have survived the recent overhaul of the academic promotions exercise.  

Open Research in the Humanities: Research Integrity and Care 

Authors: Emma Gilby, Matthias Ammon, Rachel Leow and Sam Moore

This is the fourth in a series of blog posts presenting the reflections of the Working Group on Open Research in the Humanities.  Read the opening post here. The working group aimed to reframe open research in a way that was more meaningful to humanities disciplines, and their work will inform the University of Cambridge approach to open research.   This post considers research integrity in the context of arts & humanities research.

Research integrity applies to A&H disciplines in gathering CORE data, conveying interpretations, maintaining disciplinary standards, and privileging diversity, transparency, respect, and accountability. This is ‘careful’ scholarship in its truest sense. Our conversation here took the idea of careful scholarship in two main directions, considering the labour associated with the work as process, and the labour associated with establishing and maintaining decolonial integrity. This means allowing for and legitimizing diverse voices, methods and ways of thinking.    

Opportunities 

As the Open Research conference held in November 2021 stated in its call for contributions: 

We have moved beyond the myth of the lone genius: research is a collaborative endeavour. We need to approach all stages of research more openly, to facilitate collaboration and the incremental growth of ideas. Breaking down the walls around information will enable more stakeholders, both lay and professional, to become involved and deepen their trust in research.1

In fact, the myth of the lone scholarly genius is a relatively recent phenomenon, and many of the scholarly processes in which the A&H are engaged pre-date it. The open research movement offers authors the opportunity to look beyond their own status as author, to consider the wider scholarly ecosystem, the processes behind scholarship, the networks of people involved, so that these are acknowledged openly rather than lost. As we have already stated, editing is at the heart of scholarly publishing, taking research into a legitimate, citable, creditable publication. This is particularly the case in A&H research that targets smaller scholarly communities: ‘For society publishers, where we see responsiveness to the community of researchers as mission critical, editorial work is mission central.’² 

More fundamentally, a crucial element of research integrity is tackling the need for appropriate and fair representation across a diversity of voices and communities. A major question for arts and humanities research is how to open up and take account of the global wealth of different voices – opening up to ‘fugitive’ voices that have not traditionally been archived or recognised or able to embody the ‘status’ of author in the first place. 

Support required 

As far as the existing scholarly community is concerned, editing can brought into the open: divided into the work of General Editors, who evaluate overall content and the general direction of intellectual contributions, and who make decisions about what work to accept on the basis of peer review; the work of Managing Editors, who are the chief manuscript editors and engage with the business of day-to-day communication with authors; the work of Copy Editors, who make script clear and consistent; and the work of Type Setters, who even in the digital age arrange documents for publication. All these people share care and responsibility for disciplinary standards. They also require a salary, which brings us back to the future of scholarly communications and the question of funding. Making this labour visible and public is an important way to avoid the exploitation (and self-exploitation) that is endemic in academia.  

Careful consideration here needs to be given to the issue of appropriate and fair representation across a diversity of voices and communities. Open research does not necessarily or without effort tackle the omission of voices from the public sphere, typically those of the non-white, non-male, non-cis, non-anglophone world.3 Indeed, without explicit reflection on decolonial integrity, the move towards open research paradoxically risks a homogenizing effect: allowing researchers to disseminate their research on the condition that they imitate or ventriloquize a certain subset of languages or conventions.

[1] Open Research at Cambridge 2021, call for contributions (no longer accessible online)

[2] Angela Cochran and Karin Wulf ‘Editing is at the Heart of Scholarly Publishing’, 24th April 2019, https://scholarlykitchen.sspnet.org/2019/04/24/editing-is-at-the-heart-of-scholarly-publishing/

[3] Lorena Gautherau, ‘Decolonizing the Digital Humanities’,  20th November 2017, https://recoveryprojectappblog.wordpress.com/2017/11/20/incubator-decolonizing-the-digital-humanities/; see also the article by Coker and Ozment cited above.

‘We found that Open Science policies, mostly stemming from Europe, frame “openness” as a vehicle to promote technological change as part of an inevitable and necessary cultural shift to modernity in scientific production. The global reach of these narratives, and the technologies, standards and models these narratives sustain, are dictating modes of working and collaborating among those who can access them, and creating new categories of exclusion that invalidate knowledge that cannot meet this criteria, putting historically marginalized researchers and publics at further disadvantage.’ D. Albornoz et al., ‘Framing Power: Tracing Key Discourses in Open Science Policies’, ELPUB 2018, https://dx.doi.org/10.4000/proceedings.elpub.2018.23

See also: Rebekka Kiesewetter, Undoing scholarship: Towards an activist genealogy of the OA movement, Tijdschrift voor GenderstudiesVolume 23, Issue 2, Jun 2020, p. 113 – 130

https://www.aup-online.com/content/journals/10.5117/TVGN2020.2.001.KIES The authors of ‘Labour of Love: An Open Access Manifesto for Freedom, Integrity and Creativity in the Humanities and Interpretive Social Sciences’ refer to ‘the increasingly imperiled principles of academic freedom, integrity, and creativity’.  Andrea E. Pia et al., ’Labour of Love: An Open Access Manifesto for Freedom, Integrity, and Creativity in the Humanities and Interpretive Social Sciences’, 16th July 2020, https://commonplace.knowledgefutures.org/pub/y0xy565k/release/2