Tag Archives: open access

Openness, integrity & supporting researchers

Uncategorized, , , , , ,

Universities need to open research to ensure academic integrity and adjust to support modern collaboration and scholarship tools, and begin rewarding people who have engaged in certain types of process rather than relying on traditional assessment schemes. This was the focus of Emeritus Professor Tom Cochrane’s* talk on ‘Open scholarship and links to academic integrity, reward & recognition’  given at Cambridge University on 7 October.

The slides from the presentation are available here: PRE_Cochrane_DisruptingDisincentives_V1_20151007

Benefits of an open access mandate

Tom began with a discussion about aspects of access to research and research data and why it should be as open as possible. Queensland University of Technology introduced an open access mandate 12 years or so ago. They have been able to observe a number of effects on bibliometric citation rates, such as the way authors show up in Scopus.

The other is the way consulting opportunities arise because someone’s research is exposed to reading audiences that do not have access to the toll-gated literature. Another benefit is the recruiting of HDR students.

Tom outlined six areas of advantage for institutions with a mandate – researcher identity and exposure, advantage to the institution. He noted that they can’t argue causation but can argue correlation, with the university’s. improvement in research performance. Many institutions have been able to get some advantage of having an institutional repository that reflects the output of the institution.

However in terms of public policy, the funders have moved the game on anyway. This started with private funders like Wellcome Trust, but also the public funding research councils. This is the government taxpayer argument, which is happening in the US.

Tom noted that when he began working on open access policy he had excluded books because there are challenges with open access when there is a return to the author, but there has been a problem long term with publishing in the humanities and the social sciences. He said there was an argument that there has been a vicious downward spiral that oppresses the discipline, by making the quality scholarship susceptible to judgements about sales appeal for titles in the market, assessments which may be unrelated. Now there is a new model called Knowledge Unlatched which is attempting to break this cycle and improve the number of quality long form outputs in Humanities and Social Sciences.

Nightmare scenarios

Tom started by discussing the correlation between academic integrity and research fraud by discussing the disincentives in the system. What are potential ‘nightmare’ scenarios?

For early career researcher nightmares include the PhD failing, being rejected for a job or promotion application, a grant application fails, industry or consultancy protocols fail or a paper doesn’t get accepted.

However a worse nightmare is a published or otherwise proclaimed finding is found to be at fault – either through a mistake or there is something more deliberate at play. This is a nightmare for the individual.

However it is very bad news for an institution to be on the front page news. This is very difficult to rectify.

Tom spoke about Jan Hendrik Schon’s deception. Schon was a physicist who qualified in Germany, went to work in Bell Labs in the US. He discovered ‘organic semiconductors’. The reviewers were unable to replicate the results because they didn’t have any access to the original data with lab books destroyed and samples damaged beyond recovery. The time taken to investigate and the eventual withdrawal of the research was 12.5 years, and the effort involved was extraordinary.

Incentives for institutions and researchers

Academics work towards recognition and renown, respect and acclaim. This is based on a system of dissemination and publication, which in turn is based on peer review and co-authorship using understood processes. Financial reward is mostly indirect.

Tom then discussed what structures universities might have in place. Most will have some kind of code of conduct to advise people about research misconduct. There are questions about how well understood or implemented this advice or knowledge about those kinds of perspectives actually are.

Universities also often provide teaching about authorship and the attribution of work – there are issues around the extent that student work gets acknowledged and published. Early career researchers are, or should be, advised about requirements in attributing work to others that have not contributed, as well as a good understanding of plagiarism and ethical conduct.

How does openness help?

Tom noted that we are familiar with the idea of open data and open access. But another aspect is ‘open process’. Lab work books for example, showing progress in thinking, approaches and experiments can be made open though there may be some variations in the timing of when this occurs.

The other pressing thing about this is that the nature of research itself is changing profoundly. This includes extraordinary dependence on data, and complexity requiring intermediate steps of data visualisation. In Australia this is called eResearch, in the UK it is called eScience. These eResearch techniques have been growing rapidly, and in a way that may not be understood or well led by senior administrators.

Using data

Tom described a couple of talks by early or mid career researchers at different universities. They said that when they started they were given access to the financial system, the IT and Library privileges. But they say ‘what we want to know are what are the data services that I can get from the University?’. This is particularly acute in the Life Sciences. Where is the support for the tools? What is the University doing by way of scaffolding the support services that will make that more effective for me? What sort of help and training will you provide in new ways of disseminating findings and new publishing approaches?

Researchers are notoriously preoccupied with their own time – they consider they should be supported better with these emerging examples. We need more systematic leadership in understanding these tools with a deliberate attention by institutional leadership to overcoming inertia.

The more sustained argument about things being made open relates to questions about integrity and trust – where arguments are disputes about evidence. What’s true for the academy in terms of more robust approaches to prevent or reduce inaccuracy or fraud, is also true in terms of broader public policy needs for evidence based policy.

Suggestions for improvement

We need concerted action by people at certain levels – Vice Chancellors, heads of funding councils, senior government bureaucrats. Some suggested actions for institutions and research systems at national and international levels include concerted action to:

  • develop and support open frameworks
  • harmonise supporting IP regimes
  • reframe researcher induction
  • improve data and tools support services
  • reward data science methods and re-use techniques
  • rationalise research quality markers
  • foster impact tracking in diverse tools

Discussion

Friction around University tools

One comment noted that disincentives at Cambridge University manifest as frictions around the ways they use the University tools – given they don’t want to waste time.

Tom responded that creating a policy is half the trick. Implementing it in a way that makes sense to someone is the other half. What does a mandate actually mean in a University given they are places where one does not often successfully tell someone else what to do?

However research and support tools are getting more efficient. It is a matter of marshalling the right expertise in the right place. One of the things that is happening is we are getting diverse uptakes of new ideas. This is reliant on the talent of the leadership that might be in place or the team that is in place. It could get held back by a couple of reactionary or unresponsive senior leaders. Conversely the right leadership can make striking progress.

Openness and competition

Another comment was how does openness square with researchers being worried about others finding about what they are doing in a competitive environment?

Tom noted that depending on the field, there may indeed need to be decision points or “gating” that governs when the information is available. The important point is that it is available for review for the reasons of integrity explored earlier. Exceptions will always apply as in the case of contract research being done for a company by an institution that is essentially “black box”. There would always have to be decisions about openness which would be part of working out the agreement in the first place.

Salami slicing publication

A question arose about the habit of salami slicing research into small publications for the benefits of the Research Excellence Framework and how this matches with openness.

Tom agreed that research assessment schemes need to be structured to encourage or discourage certain types of scholarly output in practice. The precursor to this practice was the branching of journal titles in the 1970s – the opportunity for advantage at the time was research groups and publishers. There has to be a leadership view from institutional management on what kind of practical limits there can be on that behaviour.

This sparked a question about the complexity of changing the reward system because researchers are judged by the impact factor, regardless of what we say to them about tweets etc. How could the reward system be changed?

Tom said the change would need to be that the view that reward is only based on research outputs is insufficient. Other research productivity needs reward. This has to be led. It can’t be a half-baked policy – put out by a committee. Needs to be trusted by the research community.

Open access drivers

A question was asked about the extent to which the compliance agenda that has been taken by the funders has led its course? Is this agenda going to be taken by the institutions.

Tom said that he has thought about this for a long time. He thought originally OA would be led by the disciplines because of the example of the High Energy Physics community which built a repository more than 20 years ago. Then there was considerable discussion, eg in the UK in early 2000s about aligning OA with institutional profile. But institutional take up was sporadic. In Australia in 2012 we only had six or seven universities with policies (which doesn’t necessarily mean there had been completely satisfactory take up in each of those).

Through that time the argument for a return on tax payer investment has become the prevalent government one. Tom doesn’t think they will move away from that, even though there has been a level of complexity relating to the position that might not have been anticipated, with large publishers keen to be embedded in process.

This moved to a question of whether this offers an opportunity for the institution beyond the mandate?

Tom replied that he always thought there was an advantage at an institutional and individual level that you would be better off if you made work open. The main commercial reaction has been for the large publishers to seek to convert the value that exists in the subscription market into the same level of value in input fees i.e, Article Processing Charges.

It should be understood finally that academic publishing and the quality certification for research does have a cost, with the question being what that level of cost should really be.

About the speaker

*Emeritus Professor Tom Cochrane was briefly visiting Cambridge from Queensland University of Technology in Australia. During his tenure as the Deputy Vice-Chancellor (Technology, Information and Learning Support), Professor Cochrane introduced the world’s first University-wide open access mandate, in January 2004. Amongst his many commitments Professor Cochrane serves on the Board of Knowledge Unlatched (UK) is a member of the Board of Enabling Open Scholarship (Europe) and was co-leader of the project to port Creative Commons into Australia.

Published 12 October 2015
Written by Dr Danny Kingsley
Creative Commons License

What is ‘research impact’ in an interconnected world?

Uncategorized, , , , , , , , , , , , ,

Perhaps we should start this discussion with a definition of ‘impact’. The term impact is used by many different groups for different purposes, and much to the chagrin of many researchers it is increasingly a factor in the Higher Education Funding Councils for England’s (HECFE) Research Excellence Framework. HEFCE defined impact as:

‘an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia’.

So we are talking about research that affects change beyond the ivory tower. What follows is a discussion about strengthening the chances of increasing the impact of research.

Is publishing communicating research?

Publishing a paper is not a good way of communicating work. There is some evidence that much published work is not read by anyone other than the reviewers. During an investigation of claims that huge numbers of papers were never cited, Dahlia Remler found that:

  • Medicine  – 12% of articles are not cited
  • Humanities – 82% of articles are not cited – note however that their prestigious research is published in books, however many books are rarely cited too.
  • Natural Sciences – 27% of articles are never cited
  • Social Sciences – 32% of articles are never cited

Hirsch’s 2005 paper: An index to quantify and individual’s scientific research output, proposing the h index – defined as the number of papers with citation number ≥h. So an h index of 5 means the author has at least 5 papers with at least 5 citations. Hirsch suggested this as a way to characterise the scientific output of researchers. He noted that after 20 years of scientific activity, an h index of 20 is a ‘successful scientist’. When you think about it, 20 other researchers are not that many people who found the work useful. And that ignores those people who are not ‘successful’ scientists who are, regardless, continuing to publish.

Making the work open access is not necessarily enough

Open access is the term used for making the contents of research papers publicly available – either by publishing them in an open access journal or by placing a copy of the work in a subject or institutional repository. There is more information about open access here.

I am a passionate supporter of open access. It breaks down cost barriers to people around the world, allowing a much greater exposure of publicly funded research. There is also considerable evidence showing that making work open access increases citations.

But is making the work open access enough? Is a 9.5MB pdf downloadable onto a telephone, or through a dail-up connection?  If the download fails at 90% you get nothing. Some publishing endeavours have recogised this as an issue, such as the Journal of Humanitarian Engineering (JHE), which won the Australian Open Access Support Group‘s 2013 Open Access Champion award for their approach to accessibility.

Language issues

The primary issue, however is the problem of understandability. Scientific and academic papers have become increasingly impenetrable as time has progressed. It’s hard to believe now that at the turn of last century scientific articles had the same readability as the New York Times.

‘This bad writing is highly educated’ is a killer sentence from Michael Billig’s well researched and written book ‘Learn to Write Badly: How to Succeed in the Social Sciences‘.  This phenomenon is not restricted to the social sciences, specialisation and a need to pull together with other members of one’s ‘tribe‘ mean that academics increasingly write in jargon and specialised language that bears little resemblance to the vernacular.

There are increasing arguments for scientific communication to the public being part of formal training. In a previous role I was involved in such a program through the Australian National Centre for the Public Awareness of Science. Certainly the opportunities for PhD students to share their work more openly have never been more plentiful. There are many three minute thesis competitions around the world. Earlier this year the British Library held a #Share your thesis competition where entrants were first asked to tweet why their PhD research is/was important using the hashtag #ShareMyThesis. The eight shortlisted entrants were asked to write a short article (up to 600 words) elaborating on their tweet and explaining why their PhD research is/was important  in an engaging and jargon-free way.

Explaining work in understandable language is not ‘dumbing it down’.  It is simply translating it into a different language. And students are not restricted to the written word. In November the eighth winner of the annual ‘Dance your PhD‘ competition sponsored by Science, Highwire Press and the AAAS will be announced.

Other benefits

There is a flow-on effect from communicating research in understandable language. In September, the Times Higher Education recently published an article ‘Top tips for finding a permanent academic job‘ where the information can be summarised as ‘communicate more’.

The Thinkable.org group’s aim is to widen the reach and impact of research projects using short videos (three minutes or less). The goal of the video is to engage the research with a wide audience. The Thinkable Open Innovation Award is a research grant that is open to all researchers in any field around the world and awarded openly by allowing Thinkable researchers and members to vote on their favourite idea. The winner of the award receives $5000 to help fund their research. This is specifically the antithesis of the usual research grant process where grants “are either restricted by geography or field, and selected via hidden panels behind closed doors”.

But the benefit is more than the prize money. This entry from a young Uni of Manchester PhD biomedical student did not win, but thousands of people engaged in her work in just few weeks of voting.

Right. Got the message. So what do I need to do?

Researcher Mike Taylor pulled together a list of 20 things a researcher needs to do when they publish a paper.  On top of putting a copy of the paper in an institutional or subject repository, suggestions include using various general social media platforms such as Twitter and blogs, and also uploading to academic platforms.

The 101 Innovations in Scholarly Communication research project run from the University of Utrecht is attempting to determine scholarly use of  communication tools. They are analysing the different tools that researchers are using through the different phases of the research lifecycle – Discovery, Analysis, Writing, Publication, Outreach and Assessment through a worldwide survey of researchers. Cambridge scholars can use a dedicated link to the survey.

There are a plethora of scholarly peer networks which all work in slightly different ways and have slightly different foci.  You can display your research into your Google Scholar or CarbonMade profile. You can collate the research you are finding into Mendeley or Zotero. You can also create an environment for academic discourse or job searching with Academia.edu, ResearchGate and LinkedIn. Other systems include Publons – a tool to register peer reviewing activity.

Publishing platforms include blogging (as evidenced here), Slideshare, Twitter, figshare, Buzzfeed. Remember, this is not about broadcasting. Successful communicators interact.

Managing an online presence

Kelli Marshall from DePaul University asks ‘how might academics—particularly those without tenure, published books, or established freelance gigs—avoid having their digital identities taken over by the negative or the uncharacteristic?’

She notes that as an academic or would-be academic, you need to take control of your public persona and then take steps to build and maintain it. If you do not have a clear online presence, you are allowing Google, Yahoo, and Bing to create your identity for you. There is a risk that the strongest ‘voices’ will be ones from websites such as Rate My Professors.

Digital footprint

Many researchers belong to an institution,  a discipline and a profession. If these change your online identity associated with them will also change. What is your long term strategy? One thing to consider is obtaining a persistent unique identifier such as an ORCID – which is linked to you and not your institution.

When you leave an institution, you not only lose access to the subscriptions the library has paid for, you also lose your email address. This can be a serious challenge when your online presence in academic social media sites like Academia.edu and ResearchGate are linked to that email address. What about content in a specific institutional repository? Brian Kelly discussed these issues at a recent conference.

We seem to have drifted a long way from impact?

The thing is that if it can be measured it will be. And digital activity is fairly easily measured. There are systems in place now to look at this kind of activity. Altmetrics.org moves beyond the traditional academic internal measures of peer review, Journal Impact Factor (JIF) and the H-index. There are many issues with the JIF, not least that it measures the vessel, not the contents. For these reasons there are now arguments such as the San Francisco Declaration on Research Assessment (DORA) which calls for the scrapping of the JIF to assess a researcher’s performance. Altmetrics.org measures the article itself, not where it is published. And it measures the activity of the articles beyond academic borders. To where the impact is occurring.

So if you are serious about being a successful academic who wants to have high impact, managing your online presence is indeed a necessary ongoing commitment.

NOTE: On 26 September, Dr Danny Kingsley spoke on this topic to the Cambridge University Alumni festival. The slides are available in Slideshare. The Twitter discussion is here.

Published 25 September 2015
Written by Dr Danny Kingsley
Creative Commons License

It’s time for open access to leave the fringe

Uncategorized, , , , , , ,
The Repository Fringe was held in Edinburgh on 3-4 August. With the theme of “Integrating repositories in the wider context of university, funder and external services”, the event brought together repository managers across the UK to discuss practice and policy. Dr Arthur Smith, Open Access Research Advisor at the University of Cambridge, attended the event and came away with the impression that more needs to be done to embed open access in scholarly processes.

In his keynote speech to Repository Fringe 2015, titled ‘Fulfilling their potential: is it time for institutional repositories to take centre stage?’  David Prosser, Executive Director of Research Libraries UK (RLUK) gave a concise overview of the history surrounding open access and the situation we currently find ourselves in, especially in the UK.

What’s become clear is that ‘we’ is a problematic term for the scholarly communications community. A lack of cohesion and vision between librarians, repository managers and administrators means ‘we’ have failed to engage with researchers to make the case for open access.

I feel this is due to, in part, the fragmented nature of repositories stemming from an institutional need for control. If national (and international) open access subject repositories had been created and exploited perhaps researcher uptake of open access in the UK and around the world would have been faster. For example, arXiv continues to be the one stop shop for physicists to publish their manuscripts precisely because it’s the repository for the entire physics community. That’s where you go if you’ve got a physics paper. To be fair, physics had a culture of sharing research papers that predates the internet.

Repositories are only as good as the content they hold, and without support from the academic community to fill repositories with content, there is a risk of side-lining green open access*. This will in turn increase the pressure to justify the cost of ineffective institutional repositories.

As David correctly identified, scholars will happily take the time to do things they feel are important. But for many researchers open access remains a low priority and something not worth investing their time in. Repositories are only capturing a fraction of their institution’s total publication output. At Cambridge we estimate that only 25-30% of articles are regularly deposited.

Providing value

The value of open access, whether it’s green or gold**, isn’t obvious to the authors producing the content. Yet juxtaposed with this is a report prepared by Nature Publishing Group on 13 August: Perceptions of open access publishing are changing for the better. This examined the changing perceptions of researchers to open access. While many researchers are still unaware of their funders’ open access requirements, the general perception of open access journals in the sciences has changed significantly, from 40% who were concerned about the quality of OA publication in 2014, to just 27% in 2015.

Clearly the trend is towards greater acceptance of open access within the academic community, but actual engagement remains low. If we don’t want to end up in a world of expensive gold open access journals, green repositories must be competitive with slick journal websites. Appearances matter. We need to attract the attention of the academics so that open access repositories are seen as viable places for disseminating research.

The scholarly communications community must find new ways of making open access (particularly green open access) appealing to researchers. One way forward is to augment the reward structure in academic publishing. Until open access is adopted more widely, academics should be rewarded for the effort involved in making their work openly available.

In the UK, failure to comply with the Higher Education Funding Council for England (HEFCE) and other funders’ policies could seriously affect future funding outcomes. It is the ever-present threat of funding cuts which drives authors to choose open access options, but this has changed open access into a policy compliance debacle.

Open access as a side effect of policy compliance is not enough; we need real support from academics to propel open access forward.

Measuring openness

As a researcher, the main things I look for when assessing other researchers and their publications are h-index, total and article level citations, and journal prestige (impact factor). I am not aware of any other methods which so simply define an author’s research.

While these types of metrics have their problems, they are nonetheless widely used within the academic community. An annual openness index, which is simply the ratio of open access articles to the total number of publications, would quickly reveal how open an academic’s research publications are. This index could be applied equally to established professors and early career researchers, as unlike the h-index, there is no historical weighting. It only depends on how you’re publishing now.

Developing such a metric would spur on open access from within academic circles by making open access publishing a competition between researchers. Perhaps the openness index could also be linked to university progression and grant reward processes. The more open access your work is, the better it is for you, and as a consequence, the community.

Open access needs to stop being a ‘fringe’ activity and become part of the mainstream. It shouldn’t be an afterthought to the publication process. Whether the solution to academic inaction is better systems or, as I believe, greater engagement and reward, I feel that the scholarly communications and repository community can look forward to many interesting developments over the coming months and years.

However, we must not be distracted from our main goal of engaging with researchers and academics to gather content for the open access repositories we have so lovingly built.

Glossary

*Green open access refers to making a copy of a published work available by placing it in a repository. This can be thought of as ‘secondary’ open access.

**Gold open access is where the research is published either in a fully open access journal – which sometimes incurs an article processing charge, or in a hybrid journal – which imposes an article processing charge to make that particular article available and also charges a subscription for the remainder of the articles in the journal. This can be thought of as ‘born’ open access.

Published 27 August 2015
Written by Dr Arthur Smith
Creative Commons License