Tag Archives: data

Methods getting their chance to shine – Apollo wants your methods!

Supporting Open Research,

By Dr. Kim Clugston, Research Data Co-ordinator, Office of Scholarly Communication

Underlying all research data is always an effective and working method and this applies across all disciplines from STEMM to the Arts, Humanities and Social Sciences. Methods are a detailed description of the tools that are used in research and can come in many forms depending on the type of research. Methods are often overlooked rather than being seen as an integral research output in their own right. Traditionally, published journals include a materials and methods section, which is often a summary due to restrictions on word limits making it difficult for other researchers to reproduce the results or replicate the study. There can sometimes be an option to submit the method as “supplementary material”, but this is not always the case. There are specific journals that publish methods and may be peer-reviewed but not all are open access, rendering them hidden behind a paywall. The last decade has seen the creation of “protocol” repositories, some with the ability to comment, adapt and even insert videos. Researchers at the University of Cambridge, from all disciplines – arts, humanities, social sciences and STEMM fields – can now publish their method openly in Apollo, our institutional repository. In this blog, we discuss why it is important to publish methods openly and how the University’s researchers and students can do this in Apollo.

The protocol sharing repository, Protocols.io, was founded in 2012. Protocols can be uploaded to the platform or created within it; they can be shared privately with others or made public. The protocols can be dynamic and interactive (rather than a static document) and can be annotated, which is ideal for highlighting information that could be key to an experiment’s success. Collaboration, adaptation and reuse are possible by creating a fork (an editable clone of a version) that can be compared with any existing versions of the same protocol. Protocols.io currently hosts nearly 16,000 public protocols, showing that there is a support for this type of platform. In July this year it was announced that Protocols.io was acquired by Springer Nature. Their press statement aims to reassure that Protocols.io mission and vision will not change with the acquisition, despite Springer Nature already hosting the world’s largest collection of published protocols in the form of SpringerProtocols along with their own version of a free and open repository, Protocol Exchange. This begs the question of whether a major commercial publisher is monopolising the protocol space, and if they are, is this or will this be a problem? At the moment there do not appear to be any restrictions on exporting/transferring protocols from Protocols.io and hopefully this will continue. This is a problem often faced by researchers using proprietary Electronic Research Notebooks (ERNs), where it can be difficult to disengage from one platform and laborious to transfer notebooks to another, all while ensuring that data integrity is maintained. Because of this, researchers may feel locked into using a particular product. Time will tell how the partnership between Protocols.io and Springer Nature develops and whether the original mission and vision of Protocols.io will remain. Currently, their Open Research plan enables researchers to make an unlimited number of protocols public, with the number of private protocols limited to two (paid plans offer more options and features).

Bio-protocol exchange (under the umbrella of Bio-protocol Journal) is a platform for researchers to find, share and discuss life science protocols with protocol search and webinars. Protocols can be submitted either to Bio-protocol or as a preprint, researchers can ask authors questions, and fork to modify and share the protocol while crediting the original author. They also have an interesting ‘Request a Protocol’ (RaP) service that searches more than 6 million published research papers for protocols or allows you to request one if you are unable to find what you are looking for. A useful feature is that you can ask the community or the original authors of the protocol any question you may have about the protocol. Bio-protocol exchange published all protocols free of charge to their authors since their launch in 2011, with substantial financial backing of their founders. Unfortunately,  it was announced that protocol articles submitted to Bio-protocol after March 1 2023 will be charged an Article Processing Charge (APC) of $1200. Researchers who do not want to pay the APC can still post a protocol for free in the Bio-protocol Preprint Repository where they will receive a DOI but will not have gone through the journal’s peer review process.

As methods are integral to successful research, it is a positive move to see the creation and growth of platforms supporting protocol development and sharing. Currently, these tend to cater for research in the sciences, and serve the important role of supporting research reproducibility. Yet, methods exist across all disciplines – arts, humanities, social sciences as well as STEMM – and we see the term ‘method’ rather than ‘protocol’ as more inclusive of all areas of research.

Apollo (Cambridge University’s repository) has now joined the growing appreciation within the research community of recognising the importance of detailing and sharing methodologies. Researchers at the University can now use their Symplectic Elements account to deposit a method into Apollo. Not only does this value the method as an output in its own right, it provides the researcher with a DOI and a publication that can be automatically updated to their ORCID profile (if ORCID is linked to their Elements account). In May this year, Apollo was awarded CoreTrustSeal certification, reinforcing the University’s commitment to preserving research outputs in the long-term and should give researchers confidence that they are depositing their work in a trustworthy digital repository.

The first method to be deposited into Apollo in this way was authored by Professor John Suckling and colleagues. Professor Suckling is Director of Research in Psychiatric Neuroimaging in the Department of Psychiatry. His published method relates to an interesting project combining art and science to create artwork that aims to represent hallucinatory experiences in individuals with diagnosed psychotic or neurodegenerative disorders. He is no stranger to depositing in Apollo; in fact, he has one of the most downloaded datasets in Apollo after depositing the Mammographic Image Analysis Society database in Apollo in 2015. This record contains the images of 322 digital mammograms from a database complied in 1992. Professor Suckling is an advocate of open research and was a speaker at the Open Research at Cambridge conference in 2021.

An interesting and exciting new platform which aims to change research culture and the way researchers are recognised is Octopus. Founded by University of Cambridge researcher Dr Alexandra Freeman, Octopus is free to use for all and is funded by UKRI and developed by Jisc. Researchers can publish instantly all research outputs without word limit constraints, which can often stifle the details. Research outputs are not restricted to articles but also include, for example, code, methods, data, videos and even ideas or short pieces of work. This serves to incentivise the importance of all research outputs. Octopus aims to level up the current skew toward publishing more sensationalist work and encourages publishing all work, such as negative findings, which are often of equal value to science but often get shelved in what is termed the ‘file drawer’ problem. A collaborative research community is encouraged to work together on pieces of a puzzle, with credit given to individual researchers rather than a long list of authors. The platform supports reproducibility, transparency, accountability and aims to allow research the best chance to advance more quickly. Through Octopus, authors retain copyright and apply a Creative Commons licence to their work; the only requirement is that published work is open access and allows derivatives. It is a breath of fresh air in the current rigid publishing structure.

Clear and transparent methods underpin research and are fundamental to the reliability, integrity and advancement of research. Is the research landscape beginning to change to allow open methods, freely published, to take centre stage and for methods to be duly recognised and rewarded as a standalone research output? We certainly hope so. The University of Cambridge is committed to supporting open research, and past and present members who have conducted research at the University can share these outputs openly in Apollo. If you would like to publish a method in Apollo, please submit it here or if you have any queries email us at info@data.cam.ac.uk.

There will be an Octopus workshop at the Open Research for Inclusion: Spotlighting Different Voices in Open Research at Cambridge on Friday 17th November 2023 at Downing College.

The Data Picture

Library and training matters, Uncategorized

I was recently named one of “the next generation of [library] leaders” as part of the CILIP 125, having been recognised as an individual who contributes energy and knowledge to improving and impacting their organisation. My area of expertise, and thus recognition, lies with the use of data within libraries. As a data analyst for the Office of Scholarly Communications at Cambridge University Library, my role focuses on empowering decisions with data driven understanding – such as supporting the Springer Nature negotiations. To develop my understanding of data, and its role within a wider organisation, further, I engage with data beyond the library – such as the Big Data London conference and the Carruthers and Jackson Data Leaders’ Summer School. Reflecting on the use of data in the wider world, what can be expected of the library and data?

The summer school provided practical advice, proven methodologies, and guidance that could apply across a variety of businesses. The course is designed to provide insight on the workflow of data officers, and their role within an organisation – no matter its stage of data maturity and literacy. Over the course of the ten weeks, leading experts discussed the role of a chief data officer (CDO), both as a business development opportunity, and as a career path for individuals. It explored the risk and governance of data within an organisation, and the final weeks focused strongly on the role of people and teams associated with data.

Peter Jackson and Caroline Carruthers addressed the differing types of CDO and described a pendulum between ‘risk aversion’ and ‘value added’. Understanding the balance between secure and proper data governance (GDPR for example) and providing value through data (such as setting up automation). The pendulum of risk to reward is relevant to many roles, including those within the library. Understanding the need to divide time and energy between creating policies and getting decision making results, is just as relevant to my role as a chief data officer. In my role I have supported decision making staff through data production, but equally, to instil a culture of data, time and energy must be dedicated to risk aversion, through tasks of researching data management, preparing training sessions for data storage, and supporting staff in data preparation.

Another important concept introduced was the DIKW pyramid – Data, Information, Knowledge, Wisdom – for understanding the value created from data. The base of the pyramid is (raw) Data, which can be processed into (useful) Information. This Information is data with meaning and a purpose and can be organised into (insightful) knowledge. Knowledge combines experiences, values, insights, and contextual information, which can then transcend to (integral) Wisdom. Wisdom is considered a deeper understanding with ethical implications and the ability to define ‘why’. The DIKW pyramid provided a frame of thought for presenting and approaching future data projects. Understanding the requirement to provide, data, information or knowledge, to better support a decision-making team.

To develop communication skills, expert Scott Taylor, known as The Data Whisperer, spoke about the three V’s for data storytelling: Vocabulary, Voice and Vision. Combining an accessible vocabulary, with a common voice will illuminate the business vision, and why that is important. This overarching concept for an organisations data approach can be scaled down to support individual data workers, to provide value – which should either grow, improve or protect the business case. Understanding how to communicate the data is a key skill as “Hardware comes and goes, software comes and goes, but data remains”. And that data that remains should be used to either grow, improve or protect the business, such that data gathered should be usable data!

At Big Data London, the organisation Women in Data hosted conversations about nurturing a culture of learning within data teams. Pulling from their experiences from minority backgrounds, the speakers highlighted the power in upskilling, sharing skills across teams and being an advocate on oneself and skills. As for what to upskill, data literacy was a hot topic across the conference. Data literacy, also called data fluency and data confidence, is the combination of ability, skills and confidence surround data and its uses. Data literacy enables more efficient work, and begs the question, what is the base level of data literacy / confidence across the library? Librarians use data daily; checking in/out material, answering students’ queries, or tracking the use of space, but are all librarians confident to use that data? This is an area I hope to explore further at the CUL, to ensure staff can use the data they have to support decisions.

Engaging with the world of data provides a big picture of the possibilities within the library. Conversations of AI (Artificial Intelligence), data policies and maturity, and shiny-new databases, software, and services, demonstrate the growing adoption of data, and therefore, libraries should follow suit. Actively taking snippets of larger conversations, developing ideas within the library space, and exploring the possibilities with data will help libraries thrive in this world of technological growth.

Lessons learned from Jisc Research Data Champions

Uncategorized, ,

In 2017 four Cambridge researchers received grants from Jisc to develop and share their research data management practices. In this blog, the four awardees each highlight one aspect of their work as a Jisc Data Champion.

The project

All four Champions embarked on a range of activities throughout the year including creating local communities interested in RDM practices, delivering training, running surveys to understand their department better, creating ‘how-to’ guides for would-be RDM mentors and testing Samvera as part of RDSS. They were excited by the freedom that the grant gave them to try out whatever RDM related activities they wanted, which meant they could develop their skills and see ideas come to fruition and make them reusable for others. For example, Annemarie Eckes developed a questionnaire on RDM practices for PhD students and Sergio Martínez Cuesta has posted his training courses on GitHub.  

However, throughout the duration of the award they also found some aspects of championing good RDM disconcerting. Whilst some sessions proved popular, others had very low attendee figures, even when a previous iteration of the session was well attended. They all shared the sense of frustration often felt by central RDM services that it is getting people to initially engage and turn up to a session that is the hard part. However, when people did come they found the sessions very useful, particularly because the Champions were able to tailor it specifically to the audience and discipline and the similar background of all the attendees provided an extra opportunity for exchanging advice and ideas that were most relevant.

The Champions tried out many different things. The Jisc Research Data Champions were expected to document and publicise their research data management (RDM) experiences and practices and contribute to the Jisc Research Data Shared Service (RDSS) development. Here the Champions each highlight one thing they tried out, which we hope will help others with their RDM engagement.

BYOD (Bring your own data)

Champion: Annemarie Eckes, PhD student, Department of Geography

The “Bring your own data” workshop was intended for anyone who thought their project data needed sorting, they needed better documentation, or even they needed to find out who is in charge or the owner of certain data. I set it up to give attendees time and space to do any kind of data-management related tasks: clean up their data, tidy up their computer/ email inbox, etc. The workshop was, really, for everyone whether at the start of their project and at the planning stage or in the middle of a project and had neglected their data management to some extent.

For the workshop the participants needed a laptop or login for the local computers to access their data and a project to tidy up or prepare, that can be done within two hours. I provided examples of file naming conventions and folder structures as well as instructions on how to write good READMEs (messages to your future self) and a data audit framework to give participants some structure to their organisation. After a brief introductory presentation about the aims and the example materials I provided, people would spend the rest of the time tidying up their data or in discussions with the other participants.

While this was an opportunity for the participants to sit down and sort out their digital files, I also wanted participants to talk to each other about their data organisation issues and data exchange solutions. Once I got everyone talking, we soon discovered that we have similar issues and were able to exchange information on very specific solutions.

1-on-1 RDM Mentoring

Champion: Andrew Thwaites, postdoc, Department of Psychology

I decided to trial 1-on-1 RDM mentoring as a way to customise RDM support for individual researchers in my department. The aim was that by the end of the 1-on-1 session, the mentee should understand how to a) share their data appropriately at the end of their project, and b) improve on their day-to-day research data management practice.

Before the meeting, I encouraged the mentee to compile a list of funders, and their funder’s data sharing requirements. During the meeting, the mentee and I would make a list of the data in the mentees project that they are aiming to share, and then I would then help them to choose a repository (or multiple repositories) to share this data on, and I’d also assist in designing the supporting documentation to accompany it. During the sessions I also had conversations about about GDPR, anonymising data, internal documentation and day-to-day practices (file naming conventions, file backups etc.) with the mentee.

As far as possible, I provided non-prescriptive advice, with the aim being to help the mentee make an informed decision, rather than forcing them into doing what I thought was best.

Embedding RDM  

Champion: Sergio Martinez Cuesta, research associate, CRUK-CI and Department of Chemistry

I came to realise early in the Jisc project that stand-alone training sessions focused exclusively on RDM concepts were not successful as students and researchers found them too abstract, uninteresting or detached from their day-to-day research or learning activities. I think the aerial view of the concept of 1-on-1 mentoring and BYOD sessions is beautiful. However, in my opinion, both strategies may face challenges with necessary numbers of mentors/trainers increasing unsustainably as the amount of researchers needing assistance grows and the research background of the audience becomes more diverse.

To facilitate take-up, I tapped into the University’s lists of oversubscribed computational courses and found that many researchers and students already shared interests in learning programming languages, data analysis skills and visualisation in Python and R. I explored how best to modify some of the already-available courses with an aim of extending the offer after having added some RDM concepts to them. The new courses were prepared and delivered during 2017-2018. Some of the observations I made were:

  • Learning programming naturally begs for proper data management as research datasets and tables need to be constantly accessed and newly created. It was helpful to embed RDM concepts (e.g. appropriate file naming and directory structure) just before showing students how to open files within a programming language.
  • The training of version control using git required separate sessions. Here students and researchers also discover how to use GitHub, which later helps them to make their code and analyses more reproducible, create their own personal research websites …
  • Gaining confidence in programming, structuring data / directories and version control in general helps students to acknowledge that research is more robust when open and contrasted by other researchers. Learning how researchers can identify themselves in a connected world with initiatives such as ORCID was also useful.

Brown Bag Lunch Seminar Series: The Productive Researcher

Champion: Melissa Scarpate, postdoc, Faculty of Education

I created the Productive Researcher seminar series to provide data management and Open Access information and resources to researchers at the Faculty of Education (FoE). The aim of the brown bag lunch format was to create an informal session where questions, answers and time for discussion could be incorporated. I structured the seminars so they covered 1) a presentation and discussion of data management and storage; 2) a presentation about Open Access journals and writing publications; 3) a presentation on grant writing where Open Access was highlighted.

While the format of the series was designed to increase attendance, the average was four attendees per session. The majority of attendees were doctoral students and postdocs who had a keen interest in properly managing their data for their theses or projects. However, I suspect it may be the case that those attending already understood data management processes and resources.

In conclusion, I think that whilst the individuals that attended these seminars found the content helpful (per their feedback) the impact of the seminars was extremely limited. Therefore, my recommendation would be to have all doctoral students take a mandatory training class on data management and Open Access topics as part of their methodological training. Furthermore, I think it may be most helpful in reaching postdocs and more senior researchers to have a mandated data management meetings with a data manager to discuss their data management and Open Access plans prior to submitting any grant proposals. Due to new laws and policies on data (GDPR) this seems a necessary step to ensure compliance and excellence in research.

Published 2 October 2018
Compiled and edited by Dr Lauren Cadwallader from contributions by Annemarie Eckes, Dr Andrew Thwaites, Dr Sergio Martinez Cuesta, Dr Melissa Scarpate
Creative Commons License

Benchmarking RDM Training

Uncategorized, , ,

This blog reports on the progress of the international project to benchmark Research Data Management training across institutions. It is a collaboration of Cambridge Research Data Facility staff with international colleagues – a full list is at the bottom of the post. This is a reblog, the original appeared on 6 October 2017. 

How effective is your RDM training?

When developing new training programmes, one often asks oneself a question about the quality of training. Is it good? How good is it? Trainers often develop feedback questionnaires and ask participants to evaluate their training. However, feedback gathered from participants attending courses does not answer the question how good was this training compared with other training on similar topics available elsewhere. As a result, improvement and innovation becomes difficult. So how to objectively assess the quality of training?

In this blog post we describe how, by working collaboratively, we created tools for objective assessment of RDM training quality.

Crowdsourcing

In order to objectively assess something, objective measures need to exist. Being unaware of any objective measures for benchmarking of a training programme, we asked Jisc’s Research Data Management mailing list for help. It turned out that a lot of resources with useful advice and guidance on creation of informative feedback forms was readily available, and we gathered all information received in a single document. However, none of the answers received provided us with the information we were looking for. To the contrary, several people said they would be interested in such metrics. This meant that objective metrics to address the quality of RDM training either did not exist, or the community was not aware of them. Therefore, we decided to create RDM training evaluation metrics.

Cross-institutional and cross-national collaboration

For metrics to be objective, and to allow benchmarking and comparisons of various RDM courses, they need to be developed collaboratively by a community who would be willing to use them. Therefore, the next question we asked Jisc’s Research Data Management mailing list was whether people would be willing to work together to develop and agree on a joint set of RDM training assessment metrics and a system, which would allow cross-comparisons and training improvements. Thankfully, the RDM community tends to be very collaborative, which was the case also this time – more than 40 people were willing to take part in this exercise and a dedicated mailing list was created to facilitate collaborative working.

Agreeing on the objectives

To ensure effective working, we first needed to agree on common goals and objectives. We agreed that the purpose of creating the minimal set of questions for benchmarking is to identify what works best for RDM training. We worked with the idea that this was for ‘basic’ face-to-face RDM training for researchers or support staff but it can be extended to other types and formats of training session. We reasoned that same set of questions used in feedback forms across institutions, combined with sharing of training materials and contextual information about sessions, should facilitate exchange of good practice and ideas. As an end result, this should allow constant improvement and innovation in RDM training. We therefore had joint objectives, but how to achieve this in practice?

Methodology

Deciding on common questions to be asked in RDM training feedback forms

In order to establish joint metrics, we first had to decide on a joint set of questions that we would all agree to use in our participant feedback forms. To do this we organised a joint catch up call during which we discussed the various questions we were asking in our feedback forms and why we thought these were important and should be mandatory in the agreed metrics. There was lots of good ideas and valuable suggestions. However, by the end of the call and after eliminating all the non-mandatory questions, we ended up with a list of thirteen questions, which we thought were all important. These however were too many to be asked of participants to fill in, especially as many institutions would need to add their own institution-specific feedback questions.

In order to bring down the number of questions which should be made mandatory in feedback forms, a short survey was created and sent to all collaborators, asking respondents to judge how important each question was (scale 1-5, 1 being ‘not important at all that this question is mandatory’ and 5 being ‘this should definitely be mandatory’.). Twenty people participated in the survey. The total score received from all respondents for each question were calculated. Subsequently, top six questions with the highest scores were selected to be made mandatory.

Ways of sharing responses and training materials

We next had to decide on the way in which we would share feedback responses from our courses and training materials themselves . We unanimously decided that Open Science Framework (OSF) supports the goals of openness, transparency and sharing, allows collaborative working and therefore is a good place to go. We therefore created a dedicated space for the project on the OSF, with separate components with the joint resources developed, a component for sharing training materials and a component for sharing anonymised feedback responses.

Next steps

With the benchmarking questions agreed and with the space created for sharing anonymised feedback and training materials, we were ready to start collecting first feedback for the collective training assessment. We also thought that this was also a good opportunity to re-iterate our short-, mid- and long-term goals.

Short-term goals

Our short-term goal is to revise our existing training materials to incorporate the agreed feedback questions into RDM training courses starting in the Autumn 2017. This would allow us to obtain the first comparative metrics at the beginning of 2018 and would allow us to evaluate if our designed methodology and tools are working and if they are fit for purpose. This would also allow us to iterate over our materials and methods as needed.

Mid-term goals

Our mid-term goal is to see if the metrics, combined with shared training materials, could allow us to identify parts of RDM training that work best and to collectively improve the quality of our training as a whole. This should be possible in mid/late-2018, allowing time to adapt training materials as result of comparative feedback gathered at the beginning of 2018 and assessing whether training adaptation resulted in better participant feedback.

Long-term goals

Our long-term goal is to collaboratively investigate and develop metrics which could allow us to measure and monitor long-term effects of our training. Feedback forms and satisfaction surveys immediately after training are useful and help to assess the overall quality of sessions delivered. However, the ultimate goal of any RDM training should be the improvement of researchers’ day to day RDM practice. Is our training really having any effects on this? In order to assess this, different kinds of metrics are needed, which would need to be coupled with long-term follow up with participants. We decided that any ideas developed on how to best address this will be also gathered in the OSF and we have created a dedicated space for the work in progress.

Reflections

When reflecting on the work we did together, we all agreed that we were quite efficient. We started in June 2017, and it took us two joint catch up calls and a couple of email exchanges to develop and agree on joint metrics for assessment of RDM training. Time will show whether the resources we create will help us meet our goals, but we all thought that during the process we have already learned a lot from each other by sharing good practice and experience. Collaboration turned out to be an excellent solution for us. Likewise, our discussions are open to everyone to join, so if you are reading this blog post and would like to collaborate with us (or to follow our conversations), simply sign up to the mailing list.

Resources

Published 9 October 2017
Written by: (in alphabetical order by surname): Cadwallader Lauren, Higman Rosie, Lawler Heather, Neish Peter, Peters Wayne, Schwamm Hardy, Teperek Marta, Verbakel Ellen, Williamson, Laurian, Busse-Wicher Marta
Creative Commons License

Milestone -1000 datasets in Cambridge’s repository

Uncategorized, , , ,

Last week, Cambridge celebrated a huge milestone – the deposit of the 1000th dataset to our repository Apollo since the launch of the Research Data Facility in early 2015. This is the culmination of a huge amount of work by the team in the Office of Scholarly Communication, in terms of developing systems, workflows, policies and through an extensive advocacy campaign. The Research Data team have run 118 events over the past couple of years and published 39 blogs.

In the past 12 months alone there have been 26000 downloads of the data in Apollo. In some cases the dataset has been downloaded many times – 170 – and the data has featured in news, blogs and Twitter.

An event was held at Cambridge University Library last week to celebrate this milestone.

   

Opening remarks

The Director of Library Services, Dr Jess Gardner opened proceedings with a speech where she noted “the Research Data Services and all who sail in her are at the core of our mission in our research library”.

Dr Gardner referred to the library’s long and proud history of collecting and managing research data that “began on vellum, paper, stone and bone”. The research data of luminaries such as Isaac Newton and Charles Darwin was on paper and, she noted “we have preserved that with great care and share it openly on line through our digital library.”

Turning to the future, Dr Gardner observed: “But our responsibility now is today’s researcher and today’s scientists and people working across all disciplines across our great university. Our preservation stewardship of that research data from the digital humanities across the biomedical is a core part of what we now do.”

“In the 21st century our support and our overriding philosophy is all about supporting open research and opening data as widely as possible,” she noted.  “It is about sharing freely wherever it is appropriate to do so”. [Dr Gardner’s speech is in full at the end of this post.]

Perspectives from a researcher

The second speaker was Zoe Adams, a PhD student at Cambridge who talked about the work she has done with Professor Simon Deakin on the Labour Regulation Index in association with the Centre for Business Research.

Ms Adams noted it was only in retrospect she could “appreciate the benefit of working in a collaborative project and open research generally”. She discussed how helpful it had been as an early career researcher to be “associated with something that was freely available”. She observed that few of her peers had many citations, and the reason she did was because “the dataset is online, people use the data, they cite the data, and cite me”.

Working openly has also improved the way she works, she explained, saying “It has given me a new perspective on what research should be about. …  It gives me a sense that people are relying on this data to be accurate and that does change the way you approach it.”

View from the team

The final speaker was Dr Lauren Cadwallader, Joint Deputy Head of the OSC with responsibility for the Research Data Facility, who discussed the “showcase dataset of the data that we can produce in the OSC” which is  taken from usage of our Request a Copy service.

Dr Cadwallader noted there has been an increase in the requests for theses over time. “This is a really exciting observation because the Board of Graduate studies have agreed that all students should deposit a digital copy of their thesis in our repository,” she said. “So it is really nice evidence that we can show our PhD students that by putting a copy in the repository people can read it and people do want to read theses in our repository.”

One observation was that several of the theses that were requested were written 60 years ago, so the repository is sharing older research as well. The topics of these theses covered algebra, Yorkshire evangelists and one of the oldest requested theses was written in 1927 about the Falkland Islands. “So there is a longevity in research and we have a duty to provide access to that research, ” she said.

Thanks go to…

The dataset itself is one created by the OSC team looking at the usage of our Request a Copy service. The analysis undertaken by Peter Sutton Long and we recently published a blog post about the findings.

The music played at the event was complied by Tony Malone and covers almost 1000 years of music, from Laura Cannell’s reworking of Hildegard of Bingen, to Jane Weaver’s Modern Cosmology. There are acknowledgments to Apollo, and Cambridge too. The soundtrack is available for those interested in listening.

This achievement is entirely due to the incredible work of the team in the Research Data Facility and their ability to engage with colleagues across the institution, the nation and the world. In particular the vision and dedication of Dr Marta Teperek cannot be understated.

In the words of Dr Gardner: “They have made our mission different, they have made our mission better, through the work they have achieved and the commitment they have.”

The event was supported by the Arcadia Fund, a charitable fund of Lisbet Rausing and Peter Baldwin.

 

 

Published 21 September 2017
Written by Dr Danny Kingsley
Creative Commons License

Speech by Dr Jess Gardner

First let us begin with some headline numbers. One thousand datasets. This is hugely significant and a very high level when looking at research repositories around the country. There is every reason to be proud of that achievement and what it means for open research.

There have been 26000 downloads of that data in the past 12 months alone – that is about use and reuse of our research data and is changing the face of how we do research. Some of these datasets have been downloaded 117 times and used in news, blogs and Twitter. The Research Data team have written 39 blogs about research data and have run 118 events, most of these have been with researchers.

While the headline numbers give us a sense of volume, perhaps let’s talk about the underlying rationale and philosophy behind this, which is core.

Cambridge University Library has a 600 year old history we are very proud of. In that time we have had an abiding responsibility to collect, care for and make available for use and reuse, information and research objects that form part of the intrinsic international scholarly record of which Cambridge has been such a strong part. And the ability for those ideas to inspire new ideas. The collection began on vellum, paper, and stone and bone.

And today much of that of course is digital. You can’t see that in the same way you can see the manuscripts and collections. It is sometimes hard to grasp when we are in this grand old dame of a building that I dare you not to love. It is home to the physical papers of such greats as Isaac Newton and Charles Darwin. Their research data was on paper and we have preserved that with great care and share it openly on line through our digital library. But our responsibility now is today’s researcher and today’s scientists and people working across all disciplines across our great university. Our preservation stewardship of that research data from the digital humanities across the biomedical is a core part of what we now do.

And the people in this room have changed that. They have made our mission different, they have made our mission better through the work they have achieved and the commitment they have.

Philosophically this is very natural extension of what we have done in the Library and the open library and its great research community for which this very building is designed. Some of you may know there is a philosophy behind this building and the famous ‘open library Cambridge’. In the 19th century and 20th century that was mostly about our open stack of books and we have quite a few of them, we are a little weighed down by them.

Our research data weighs less but it is just as significant and in the 21st century our support and our overriding philosophy is all about supporting open research and opening data as widely as possible. It is about sharing freely wherever it is appropriate to do so and there are many reasons why data isn’t open sometimes, and that is fine. What we are looking for is managing so we can make those choices appropriately, just as we have with the archive for many, many years.

So whilst as there is a fantastic achievement to mark tonight with those 1000 datasets it really is significant, we are really celebrating a deeper milestone with our research partners, our data champions, our colleagues in the research office and in the libraries across Cambridge, and that is about the changing role in research support and library research support in the digital age, and I think that is something we should be very proud of in terms of what we have achieved at Cambridge. I certainly am.

I am relatively new here at Cambridge. One of the things that was said to me when I was first appointed to the job was how lucky I was to be working at this University but also with the Office of Scholarly Communication in particular and that has proved to be absolutely true. I like to take this opportunity to note that achievement of 1000 datasets and to state very publicly that the Research Data Services and all who sail in her are at the core of our mission in our research library. But also to thank you and the teams involved for your superb achievements. It really is something to be very proud of and I thank you.

 

Sustaining long-term access to open research resources – a university library perspective

Uncategorized, ,

In the third in a series of three blog posts, Dave Gerrard, a Technical Specialist Fellow from the Polonsky-Foundation-funded Digital Preservation at Oxford and Cambridge project, describes how he thinks university libraries might contribute to ensuring access to Open Research for the longer-term.  The series began with Open Resources, who should pay, and continued with Sustaining open research resources – a funder perspective.

Blog post in a nutshell

This blog post works from the position that the user-bases for Open Research repositories in specific scientific domains are often very different to those of institutional repositories managed by university libraries.

It discusses how in the digital era we could deal with the differences between those user-bases more effectively. The upshot might be an approach to the management of Open Research that requires both types of repository to work alongside each other, with differing responsibilities, at least while the Open Research in question is still active.

And, while this proposed method of working together wouldn’t clarify ‘who is going to pay’ entirely, it at least clarifies who might be responsible for finding funding for each aspect of the task of maintaining access in the long-term.

Designating a repository’s user community for the long-term

Let’s start with some definitions. One of the core models in Digital Preservation, the International Standard Open Archival Information System Reference Model (or OAIS) defines ‘the long term’ as: 

“A period of time long enough for there to be concern about the impacts of changing technologies, including support for new media and data formats, and of a changing Designated Community, on the information being held in an OAIS. This period extends into the indefinite future.”

This leads us to two further important concepts defined by the OAIS:

“Designated Communities” are an identified group of potential Consumers who should be able to understand a particular set of information”, i.e. the set of information collected by the ‘archival information system’. 

A “Representation Information Network” is the tool that allows the communities to explore the metadata which describes the core information collected. This metadata will consist of:

  • descriptions of the data contained in the repository
  • metadata about the software used to work with that data,
  • the formats in which the data are stored and related to each other, and so forth.  

In the example of the Virtual Fly Brain Platform repository discussed in the first post in this series, the Designated Community appears to be: “… neurobiologists [who want] to explore the detailed neuroanatomy, neuron connectivity and gene expression of Drosophila melanogaster.” And one of the key pieces of Representation Information, namely “how everything in the repository relates to everything else”, is based upon a complex ontology of fly anatomy.

It is easy to conclude, therefore, that you really do need to be a neurobiologist to use the repository: it is fundamentally, deeply and unashamedly confusing to anyone else that might try to use it.

Tending towards a general audience

The concept of Designated Communities is one that, in my opinion, the OAIS Reference Model never adequately gets to grips with. For instance, the OAIS Model suggests including explanatory information in specialist repositories to make the content understandable to the general community.

Long term access within this definition thus implies designing repositories for Designated Communities consisting of what my co-Polonsky-Fellow Lee Pretlove describes as: “all of humanity, plus robots”. The deluge of additional information that would need to be added to support this totally general resource would render it unusable; to aim at everybody is effectively aiming at nobody. And, crucially, “nobody” is precisely who is most likely to fund a “specialist repository for everyone”, too.

History provides a solution

One way out of this impasse is to think about currently existing repositories of scientific information from more than 100 years ago. We maintain a fine example at Cambridge: The Darwin Correspondence Project, though it can’t be compared directly to Virtual Fly Brain. The former doesn’t contain specialist scientific information like that held by the latter – it holds letters, notebooks, diary entries etc – ‘personal papers’ in other words. These types of materials are what university archives tend to collect.

Repositories like Darwin Correspondence don’t have “all of humanity, plus robots” Designated Communities, either. They’re aimed at historians of science, and those researching the time period when the science was conducted. Such communities tend more towards the general than ‘neurobiologists’, but are still specialised enough to enable production and management of workable, usable, logical archives.

We don’t have to wait for the professor to die any more

So we have two quite different types of repository. There’s the ‘ultra-specialised’ Open Research repository for the Designated Community of researchers in the related domain, and then there’s the more general institutional ‘special collection’ repository containing materials that provide context to the science, such as correspondence between scientists, notebooks (which are becoming fully electronic), and rough ‘back of the envelope’ ideas. Sitting somewhere between the two are publications – the specialist repository might host early drafts and work in progress, while the institutional repository contains finished, publish work. And the institutional repository might also collect enough data to support these publications, too, like our own Apollo Repository does.

The way digital disrupts this relationship is quite simple: a scientist needs access to her ‘personal papers’ while she’s still working, so, in the old days (i.e. more than 25 years ago) the archive couldn’t take these while she was still active, and would often have to wait for the professor to retire, or even die, before such items could be donated. However, now everything is digital, the prof can both keep her “papers” locally and deposit them at the same time. The library special collection doesn’t need to wait for the professor to die to get their hands on the context of her work. Or indeed, wait for her to become a professor.

Key issues this disruption raises

If we accept that specialist Open Research repositories are where researchers carry out their work, that the institutional repository role is to collect contextual material to help us understand that work further down the line, then what questions does this raise about how those managing these repositories might work together?

How will the relationship between archivists and researchers change?

The move to digital methods of working will change the relationships between scientists and archivists.  Institutional repository staff will become increasingly obliged to forge relationships with scientists earlier in their careers. Of course, the archivists will need to work out which current research activity is likely to resonate most in future. Collection policies might have to be more closely in step with funding trends, for instance? Perhaps the university archivist of the digital future might spend a little more time hanging round the research office?

How will scientists’ behaviour have to change?

A further outcome of being able to donate digitally is that scientists become more responsible for managing their personal digital materials well, so that it’s easier to donate them as they go along. This has been well highlighted by another of the Polonsky Fellows, Sarah Mason at the Bodleian Libraries, who has delivered personal digital archiving training to staff at Oxford, in part based on advice from the Digital Preservation Coalition. The good news here is that such behaviour actually helps people keep their ongoing work neat and tidy, too.

How can we tell when the switch between Designated Communities occurs?

Is it the case that there is a ‘switch-over’ between the two types of Designated Community described above? Does the ‘research lifecycle’ actually include a phase where the active science in a particular domain starts to die down, but the historical interest in that domain starts to increase? I expect that this might be the case, even though it’s not in any of the lifecycle models I’ve seen, which mostly seem to model research as either continuing on a level perpetually, or stopping instantly. But such a phase is likely to vary greatly even between quite closely-related scientific domains. Variables such as the methods and technologies used to conduct the science, what impact the particular scientific domain has upon the public, to what degree theories within the domain conflict, indeed a plethora of factors, are likely to influence the answer.

How might two archives working side-by-side help manage digital obsolescence?

Not having access to the kit needed to work with scientific data in future is one of the biggest threats to genuine ‘long-term’ access to Open Research, but one that I think it really does fall to the university to mitigate. Active scientists using a dedicated, domain specific repository are by default going to be able to deal with the material in that repository: if one team deposits some material that others don’t have the technology to use, then they will as a matter of course sort that out amongst themselves at the time, and they shouldn’t have to concern themselves with what people will do 100 years later.

However, university repositories do have more of a responsibility to history, and a daunting responsibility it is. There is some good news here, though… For a start, universities have a good deal of purchasing power they can bring to bear upon equipment vendors, in order to insist, for example, that they produce hardware and software that creates data in formats that can be preserved easily, and to grant software licenses in perpetuity for preservation purposes.

What’s more fundamental, though, is that the very contextual materials I’ve argued that university special collections should be collecting from scientists ‘as they go along’ are the precise materials science historians of the future will use to work out how to use such “ancient” technology.

Who pays?

The final, but perhaps most pressing question, is ‘who pays for all this’? Well – I believe that managing long-term access to Open Research in two active repositories working together, with two distinct Designated Communities, at least might makes things a little clearer. Funding specialist Open Research repositories should be the responsibility of funders in that domain, but they shouldn’t have to worry about long-term access to those resources. As long as the science is active enough that it’s getting funded, then a proportion of that funding should go to the repositories that science needs to support it. The exact proportion should depend upon the value the repository brings – might be calculated using factors such as how much the repository is used, how much time using it saves, what researchers’ time is worth, how many Research Excellence Framework brownie points (or similar) come about as a result of collaborations enabled by that repository, etc etc.

On the other hand, I believe that university / institutional repositories need to find quite separate funding for their archivists to start building relationships with those same scientists, and working with them to both collect the context surrounding their science as they go along, and prepare for the time when the specialist repository needs to be mothballed. With such contextual materials in place, there don’t seem to be too many insurmountable technical reasons why, when it’s acknowledged that the “switch from one Designated Community to another” has reached the requisite tipping point, the university / institutional repository couldn’t archive the whole of the specialist research repository, describe it sensibly using the contextual material they have collected from the relevant scientists as they’ve gone along, and then store it cheaply on a low-energy medium (i.e. tape, currently). It would then be “available” to those science historians that really wanted to have a go at understanding it in future, based on what they could piece together about it from all the contextual information held by the university in a more immediately accessible state.

Hence the earlier the institutional repository can start forging relationships with researchers, the better. But it’s something for the institutional archive to worry about, and get the funding for, not the researcher.

Published 11 September 2017
Written by Dave Gerrard
Creative Commons License

Open at scale: sharing images in the Open Research Pilot

Uncategorized, , ,

Dr Ben Steventon is one of the participants in the Open Research Pilot. He is working with the Office of Scholarly Communication to make his research process more open and here reports on some of the major challenges he perceives at the beginning of the project.

The Steventon Group is a new group established last year which looks at embryonic development, in particular focusing on the zebrafish. To investigate problems in this area the group uses time-lapse imaging and tracks cells in 3D visualisations which presents many challenges when it comes to data sharing, which they hope to address through the Wellcome Trust Open Research Project. Whilst the difficulties that this group are facing are specific to a particular type of research, they highlight some common challenges across open research: sharing large files, dealing with proprietary software and joining up the different outputs of a group.

Sharing imaging data 

The data created by time-lapse imaging and cell tracking is frequently on a scale that presents a technical, as well as financial, challenge. The raw data consists of several terabytes of film which is then compressed for analysis into 500GB files. These compressed files are of a high enough quality that they can be used for analysis but they are still not small enough that they can be easily shared. In addition the group also generates spreadsheets of tracking data, which can be easily shared but are meaningless without the original imaging files and specific software to allow the two pieces of data to be connected. One solution which we are considering is the Image Data Resource, which is working to make imaging datasets in the life sciences, which have not previously been shareable due to their size, available to the scientific community to re-use.

Making it usable

The software used in this type of research is a major barrier to making the group’s work reproducible. The Imaris software the group uses costs thousands of pounds so anything shared in their proprietary formats are only accessible to an extremely small group of researchers at wealthier institutions, which is in direct opposition to the principles of Open Research. It is possible to use Fiji, an open source alternative, to recreate tracking with the imaging files and tracking spreadsheets; however, the data annotation originally performed in Imaris will be lost when the images are not saved in the proprietary formats.

An additional problem in such analyses is the sharing of protocols that detail the methodologies applied, from the preparation of the samples all the way through data generation and analysis. This is a common problem with standard peer-review journals that are often limited in the space available for the description of methods. The group are exploring new ways to communicate their research protocols and have created an article for the Journal of Visualised Experiments, but these are time consuming to create and so are not always possible. Open peer-review platforms potentially offer a solution to sharing detailed protocols in a more rapid manner, as do specialist platforms such as Wellcome Open Research and Protocols.io.

Increasing efficiency by increasing openness

Whilst the file size and proprietary software in this type of research presents some barriers to sharing, there are also opportunities through sharing to improve practice across the community. Currently there are several different software packages being used for visualisation and tracking. Therefore, sharing more imaging data would allow groups to try out different types of images on different tools and make better purchasing decisions with their grant money. Furthermore, there is a great frustration in this area that lots of people are working on different algorithms for different datasets, so greater sharing of these algorithms could reduce the amount of time wasted creating algorithms when it might be possible to adapt a pre-existing one.

Shifting models of scholarly communication

As we move towards a model of greater openness, research groups are facing a new difficulty in working out how best to present their myriad outputs. The Steventon group intends to publish data (in some form), protocols and a preprint at the same time as submitting their papers to a traditional journal. This will make their work more reproducible, and it also allows researchers who are interested in different aspects of their work to access the bits that interest them. These outputs will link to one another, through citations, but this relies on close reading of the different outputs and checking references. The Steventon group would like to make the links between the different aspects of their work more obvious and browsable, so the context is clear to anyone interest in the lab’s work. As the research of the group is so visual it would be appropriate to represent the different aspects of their work in a more appealing form than a list of links.
The Steventon lab is attempting to link and contextualise their work through their website, and it is possible to cross-reference resources in many repositories (including Cambridge’s Apollo), but they would like there to be a more sustainable solution. They work in areas with crossovers to other disciplines – some people may be interested in their methodologies, others the particular species they work on, and others still the particular developmental processes they are researching. There are opportunities here for openness to increase the discoverability of interdisciplinary research and we will be exploring this, as well as the issues around sharing images and proprietary software, as part of the Open Research Pilot.

Published 8 May 2017
Written by Rosie Higman and Dr Ben Steventon

Creative Commons License

‘Paperless research’ solutions – Electronic Lab Notebooks

Uncategorized, ,

The Office of Scholarly Communication started 2017 with a discussion about ‘going digital’ – on 13 January 2017 we organised an event at Cambridge University’s Department of Engineering to flesh out the problems preventing researchers from implementing Electronic Lab Notebook solutions. Chris Brown from Jisc wrote an excellent blog post with his reflections of the event* and agreed for us to re-blog it here.

For researchers working in laboratories the importance of recording experiments, results, workflows, etc in a notebook is engrained into you as a student. However, these paper-based solutions are not ideal when it comes to sharing and preservation. They pile on desks and shelves, vary in quality and often include printed data stuck in. To improve on this situation and resolve many of these issues, e-lab notebooks (ELNs) have been developed. Jisc has been involved in this work through funding projects such as CamELN and LabTrove in the past. Recently, interest in this area has been renewed with the Next Generation Research Environment co-design challenge.

On Friday 13 January I attended the E-Lab Notebooks workshop at the University of Cambridge, organised by Office of Scholarly Communication. Its purpose was to open up the discussion about how ELNs are being used in different contexts and formats, and the concerns and motivations for people working in labs. A range of perspectives and experience was given through presentations, group and panel discussions. The audience were mostly from Cambridge, but there was representation from other parts of the UK, as well as Denmark and Germany. A poll at the start showed that the majority of the audience were researchers (57%).

Institutional and researchers’ perspective on ELNs at Cambridge

The first part of the workshop focussed on the practitioners’ perspective with presentations from the School of Biological Sciences. Alastair Downie (Gurdon Institute) talked about their requirements for an ELN as well as anxieties and risks of adopting a particular system. Research groups currently use a variety of tools, such as Evernote and Dropbox, and often these are trusted more than ELNs. The importance of trust frequently came up during the day. Alastair conducted a survey to gather more detail on the use and requirements of ELNs and received an impressive 345 responses. Cost and complexity were given as the main reasons not to use ELNs. However, when asked for the most important features, cost was less important but ease of use was the most. Researchers want training, voice recognition and remote access. There is clear interest across the school at all levels, but it requires a push with guidance and direction.

Pic1Marko Hyvönen (Dept of Biochemistry) gave the PI perspective and the issues with an ELN for a biochemical lab. He reinforced what Alastair had said about ELNs. He showed how paper log books pile up, deteriorate over time and sometimes include printed information. They are hard to read and easy to destroy, a poor return on effort, often disappear and not searchable. It was interesting to hear about bad habits such as storing data in non-standardised ways, missing data, printing out Word documents and sticking them into the lab books.

With 99% of their data electronic many of the issues in the use of lab books generally are around data management and not ELNs. An ELN solution should be easy to use, cross platform, have a browser front end, be generic/adaptable, allow sharing of data and experiments, enforce Standard Operating Procedures when needed, have templates for standard work to minimise repetition, include inputting of data from phones and other non-specific devices. What they don’t want are the “bells and whistles” features they don’t use. Getting buy-in from people is the top issue to overcome in implementing an ELN.

Views on ELNs from outside the UK

Jan Krause from the École pPolytechnique Fédérale de Lausanne (EPFL) gave a non-UK perspective on ELNs. He described a study, as part of a national RDM project, where they separated ELNs (75 proprietary, 12 open source – 91 features) and Lab Info Management Systems (LIMS) (281 proprietary, 9 open source – 95 features) and compared their features. The two tools used mostly in Switzerland are SLims (commercial solution) and openBIS (homemade tool). To decide which tool to use they undertook a three phase selection process. The first selection was based on disciplinary and technical requirements. The second selection involved detailed analysis based on user requirements (interviews and evaluation weighted by feature) and price. The third selection was tendering and live demos.

Data storage, security and compliance requirements

When using and sharing data you need to make sure your data is safe and secure. Kieren Lovell, from the University Information Services, talked about how researchers should keep their data and accounts safe. Since he started in May 2015, all successful hacks on the university have been due to human error, such as unpatched servers, failures in processes, bad password management, and phishing. Even if you think your data and research isn’t important, the reputational damage of security attacks to the university is huge. He recommended that any research data is shared through cloud providers rather than email, never trust public wifi as is not secure so use Cambridge’s VPN service. If using a local machine you should encrypt your hard drive.

Pic2

Providers’ perspective

In the afternoon, presentations were from the providers’ perspective. Jeremy Frey, from the University of Southampton, talked about his experience of developing an open source ELN to support open and interdisciplinary science. He works on getting the people and technology to work together. It’s not just recording what you have done, you need to include the narrative behind what you do. This is critical for understanding and ELNs are one part of the digital ecosystem in the lab. The solution they’ve developed is LabTrove, partly funded by Jisc, which is a flexible open source web based solution. Allowing pictures to be added to the notes has really helped with accessibility and usability, such as dyslexia. Sustainability, as is often the case, came up and how a community is required to support such a system. It also needs to expand beyond Southampton. Finally, Jeremy used Amazon Echo to query the temperature within part of his lab. He hopes that this will be used more in the lab in the future when it can recognise each researcher’s voice.

In the next two presentations, it was over to the vendors to show the advantages of adopting RSpace (by Rory Macneil) and Dotmatics (by Dan Ormsby). The functionality on offer in these types of solutions is attractive for scientists and RSpace showed how it links to most common file stores. With any ELN, it should enhance researchers’ workflow and integrate with the tools they use.

Removing the barriers

After lunch there were three parallel focus group discussions. I attended the one on sustainability, something that comes up frequently in discussions, particularly when looking at open source or proprietary solutions. Each group reported back as follows:

Focus group 1: Managing the supplier lock in risk

Stories of use need to be shared. The PDF is not a great format for sharing. Vendors tell the truth like estate agents. Have to accept the reality that won’t have 100% exporting functionality so need to decide the minimum level. Determine specific users’ requirements.

Focus group 2: Sustainability of ELN solutions

What is the lifetime of an ELN? How long should everything be accessible? Various needs come from group and funder requirements, e.g. 10 years. There is concern if you are relying on one commercial solution as companies can die, so how can you guarantee the data will be available? Have exit policies and support standards and interoperability so data can be moved across ELNs. Broken links and file formats expiring is not just an ELN problem, but relates to the archiving of data in general. Should selection and support of an ELN be at group, department, institution or national level? This is difficult if it’s in one group as adopting any technical solution requires support in place. It requires institutional level support.

Focus group 3: Human element of ELN implementation

The biggest hurdle is culture change and showing the benefits of using an ELN. Training and technical support costs money and time. It would cost more initially but becomes more efficient. You can incentivise people by having champions. There are different needs in a large institution. You may join a lab and find the ELN is not adequate. Legal issues around sensitive data complicates matters. You need to believe it will save time. Long term solutions include using cloud base solutions, even MS Office, but what happens when people leave? Need support from higher level. Functionality should be based on user requirements. A start would be to set up a mailing list of people interested in ELNs.

Remaining barriers to wide ELN adoption

Finally, I chaired a panel session with all the presenters. Marta Teperek had kindly asked me to give a short presentation on what Jisc does as many researchers don’t know (in fact I was asked “what’s Jisc?” in the focus group) and to promote the Next Generation Research Environment co-design challenge. Following my presentation the discussion was prompted by questions from the audience and remotely via sli.do. Much of the discussion re-iterated what had been said in the presentations, such as the importance of an ELN that meets the requirements of researchers. It should allow integration with other tools and exporting of the data for use it other ELNs. Getting ELNs used within a department is often difficult so it does need institution level commitment and support. Without this ELNs are unlikely to be adopted within an institution, never mind nationally. One size does not fit all and we should not try to build an ELN that tries to satisfy the different needs of various disciplines. A modular system that integrates with the tools and systems already in use would be a better solution. Much of what was said tallied with the feedback received for the Next Generation Research Environment co-design challenge.

Closing remarks

Ian Bruno closed the workshop and he reiterated what was said in the panel discussion. I found the event extremely helpful and it provided lots of useful information to feed into the Next Generation Research Environment work. I’d like to thank Marta Teperek for inviting me to chair the panel and for all her hard work putting the event together with @CamOpenData. Marta has put together the tweets from the day into the following storify.  All notes and presentations from the event are now published in Apollo, the University of Cambridge’s research repository.

Follow-up actions at the University of Cambridge – give it a go!

Those of you who are interested in ELNs and who are based at the University of Cambridge might be interested in knowing that we are planning to do some trial access to Electronic Lab Notebooks (ELN). The purpose of this trial will be to test out several ELNs to decide on solutions which might best meet the requirements of the research community. A mailing list has been set up for people who are interested in being part of this pilot or would like to be involved in these discussions. If you would like to be added to the mailing list, please fill in the form here: https://lists.cam.ac.uk/mailman/listinfo/lib-eln

*Originally published by Jisc on 18 January 2017.

Published on 29 January 2017
Written by Chris Brown
Creative Commons License

Press embargoes – a threat from the shadows

Uncategorized, , , , ,

Something has been rumbling under the surface in the repository world recently, at least in the UK. Over the past six months or so, the Office of Scholarly Communication has had some fraught conversations with researchers who are terrified that their papers will be ‘pulled’ from publication by the journal. The reason is because some information about the upcoming paper is publicly available.

The HEFCE policy asks us to deposit the Author’s Accepted Manuscript into a repository  “as soon after the point of acceptance as possible” and to present the manuscript “in a way that allows it to be discovered by readers and by automated tools such as search engine”.  So when a researcher deposits a paper we check the publisher copyright restrictions and deposit the work into the repository – shutting down the article, but making the record public. What this means is that the metadata about an article is publicly available before it is published.

Similarly, researchers also share their research data in the repository and make the metadata publicly available. Funders and universities require researchers to include a statement in their publications about the availability of research data supporting their findings. What this means in practice is that researchers deposit research data supporting their manuscripts into data repositories before the corresponding manuscripts are published and more and more frequently, even before these are accepted (to ensure that peer reviewers have access to all supporting materials).

Terminology

Before I go into detail about our challenge, let’s get a few terms straight here. When we talk about ‘data’ in this space we mean the information generated during a research project from which observations and conclusions are made in academic papers. The ‘metadata’ is the information about that data. In the case of an article, the metadata includes the authors, the title, the journal and the abstract. The metadata for datasets is less well defined, although there are data citation principles, so metadata in this case is information about what the data is, how it was generated, how to access and interpret it.

The second term that matters here is ’embargo’ – the word of the moment for me, given my recent participation in the Open Scholarship Initiative Embargo Workgroup.  Our resolution was that we need to fund a research piece to resolve the reasoning behind embargoes (bearing in mind there is no link between half life of article usage and subscription rates) and what is a reasonable period of time for them.

Some publishers impose publication embargoes on the release of the Author’s Accepted Manuscript for a period of time after publication. The business of managing embargoes falls into the laps of repository managers. Indeed I have written before about how the complexity of different publisher agreements means it is almost impossible for an individual researcher to navigate the rules. We manage embargoes by putting the article under indefinite embargo when we deposit it and then check back on a rotational basis to see if the work has been published. Once it has, we can set the embargo date. And this is time consuming – to give an idea of scale we currently have over 1700 papers in the ‘checking’ pile.

There is a second type of embargo – a press embargo. This is an embargo which prohibits anyone actively discussing the content of accepted papers with the media prior to publication. The exception is that a few days before publication the journal allows journalists access to the published papers so they can prepare news stories to coincide with the publication of the work. It is this second type of embargo that is causing confusion with the research community.

The perceived problem

Our researchers are concerned that having the metadata about an article available means that publishers will consider this a breach of embargo and will pull the publication. Note that the Author’s Accepted Manuscript of the article itself (or the data files, in case of datasets) is locked down and the information about the volume, issue and pages are missing as the work is not yet published.

The researchers are worried because there is a need for publication in high profile journals such as Nature for their careers and if a work was to be pulled from publication this would have huge implications for them. This has caused a challenge for us – clearly we do not wish to threaten our researchers’ publication prospects, but we are also bound by the requirements of the HEFCE policy.

In November last year I put a query out on a couple of mail lists about this which generated a great deal of discussion. As it happens some UK repository managers are locking down the metadata about ALL articles until publication.

The actual problem

So we have decided to go to the source and are now in discussion with various publishers. A few things have come to light. First, generally publishers understand the distinction between these embargoes (details about responses are below). However there appears to be confusion at the editor level about this – and our researchers are in contact with editors.

The second issue is the level of bullying that researchers appear to be subjected to by the publishing industry. They are petrified of doing the ‘wrong thing’, and that they will be punished by having their article pulled. There is of course a question about whether any articles ever have actually been pulled because of metadata being available prior to publication or if this is yet another ghost we are boxing. No researchers to date have given us a concrete example of this happening.

Below is some of the correspondence we have had with publishers to date on this issue. The responses have been varied – from helpful and encouraging to restrictive and uninformative.

No issue – BMJ

The University of York had some information back from BMJ, which noted that the self archiving policy says that the post-print – “Final draft of manuscript: post peer-review, before the article is copyedited, typeset and published” – can be made available without embargo.  They did note that the press (not the author) is subject to the rule “All material accepted for publication in any BMJ journal is under embargo until it is published online.”

Permission and concern – Nature Publishing Group

Late last year I contacted Nature Publishing Group (NPG) to ask their position on this issue and they were adamant that they do not pull papers from publication if the metadata is available prior to publication. They did: “ask that our two embargoes – self-archiving embargo and press embargo – are respected”. They clarified that “NPG deposition to all PMC repositories allows the deposits to be fully discoverable as soon as processed by the repository, and the manuscripts’ full text become accessible 6 months after publication. In practice, this also means metadata can be available upon acceptance“.

NPG also looked at the question of articles being “pulled” ahead of publication and expressed concern that this idea was being propagated. They said that “to our knowledge there are no cases in which this has happened by putting metadata in a university repository. If you have information about any cases in which this is claimed to be the case, we would be very grateful if they could be sent through to us so that we can investigate them further.”

Poor practice – Science

We recently had a Science paper where we had already deposited the data associated with that paper into our repository, (we had shut the data down for release on publication date) and had generated a link that the researchers were able to include in their paper. However the issue of having information in the public domain prior to publication was raised by our researchers so we wrote to Science for clarification. Their response was:

“I discussed your question with my editorial colleagues, and our provisional response is that we would prefer you to take the cautious option of keeping the metadata in the ‘dark archive’ until the date of publication. We appreciate, though, that this is non-ideal from your point of view, and will now be discussing the question with our Office of Public Programs in Washington, to see whether we can accommodate your preferred procedure.”

Remember that this is the publisher telling us to suppress the metadata about data that is published by us in our repository.

The big problem we have (apart from the principle of this issue) is that while we can automate the turning off of an embargo in the repository we cannot automate the movement of an item from a dark archive to an open one – this must be done manually. The paper was published on Good Friday. There was no-one physically at work (indeed the Library was closed) until the following Tuesday. So for the first four days of this article being in the public domain there was a dead link to data in it. This is not just ‘non-ideal’ – it is contrary to the idea of effective and complete publication.

Communication breakdown – The Lancet

An example of the problem we are encountering  happened only last week. We were contacted by a researcher who demanded we take down the details of their accepted paper that was to be published in The Lancet because an assistant editor had taken issue with us posting the metadata prior to publication. Again we contacted the publisher and asked them their position.

When we spoke to The Lancet they were helpful and positive, reassuring us they were “happy to permit the release of article metadata at this stage”. They also said they had been in contact with the assistant editor of this article who has directly contacted the researcher. The explanation was “it looks like there was a misunderstanding regarding the wording of one of our policies”.

Right back at ya – Elsevier

Our correspondence with Elsevier was particularly unhelpful. While acknowledging that press embargoes are “editorial policy rather than open access policy” and that they are designed to create press interest, they concluded that this “isn’t a matter for a corporate policy at company level”.

This means it is apparently up to us to find out individual journal positions rather than the organisation taking responsibility for what is becoming a major problem for us. We contacted Cell Press (a subsidiary of Elsevier). They replied:

“You’ll notice that our policy is that we discourage release of the metadata and abstract prior to official publication. The reason is that this would be considered breaking any press embargoes on the article. So while, like Nature, we would not prohibit release of metadata, it means that it would be unlikely that the article would be considered for a press release. If you think that your article is likely to be appropriate for a press release to top-tier media outlets, then we recommend delaying deposit to the institutional repository until the article is officially published.

So not only do Elsevier not give us an answer, and we need to contacts journals directly, we now have to make editorial decisions on individual Cell Press articles to determine if these might be potentially worthy of a press release. Now, as it happens I have a science journalism background* and could possibly do this – but are we seriously required in the UK to employ repository managers with news reporting skills so they can concurrently meet HEFCE requirements and also ensure that the public profile of their institution’s research is protected? (*This means, ironically, that I have many years of direct experience with press embargoes.)

There are two problems here. There is the practical embargo issue repository managers face, and there is the bullying of researchers problem.

Protecting researchers

In the case with The Lancet, despite us explaining that this was a considerably larger problem for us than this particular example, the researcher was unwilling to give us any further details as to the communication they had received from the journal other than it had been an ‘assistant editor’. This ‘protection’ of journals by researchers is not uncommon in our experience. indeed, a recent article noted the “oppression” by editors stating for example that researchers are “afraid to say anything about the New England Journal because they’re afraid they won’t get something published there”.

Bear in mind that Nature specifically stated that they were unaware of any examples of a paper being pulled because the metadata was available prior to publication. We have not been able to uncover a single concrete example of this actually occurring. Yet we continue to have distressed and frightened researchers contacting us because of this threat.

I don’t have an answer to this bullying problem other than our need to move away from having publication in a high impact journal as the be-all and end-all for research careers. In case you are in any doubt of how destructive this situation is, consider ‘Your Right Arm for a Publication in AER?‘ where researchers indicated they would be prepared to lose half a thumb for a high profile publication. But reconfiguring the reward and recognition processes in research is a long way off. Meanwhile the least we can do is acknowledge this is happening and bring it into the open.

Clarity on press embargoes

In relation to the embargo issue, while we can continue to contact each publisher in turn and try and negotiate these issues with them, this is hardly time efficient given it is a sector-wide problem. The ‘solution’ from some (well resourced and staffed) publishers is that it is up to individual repository managers  to contact each journal in turn for clarification. This is clearly not possible given that our staff time is already spent checking and complying with the myriad of different publisher requirements in place.

A different solution could be that publishers shorten the time between acceptance and publication. When journals have a quick turnover time, the period when an article’s metadata is publicly available before publication is limited to a matter of days. It is different when there is a delay of months or even years between acceptance and publication.

I should note that our team debated about whether writing this piece would actually do more harm than good – triggering publishers to suddenly introduce further restrictions on metadata. Managing repositories and juggling embargoes and funder policies is already complicated enough. Material available in institutional repositories is only a small percentage of material that is publicly available – according to the Monitoring the Transition to Open Access report, institutional repositories in the UK hold 7.9% of Author Accepted Manuscripts, compared to 56.6% in subject repositories and 19.7% in social sharing sites. Yet we are generally the ones actually adhering to the complex sets of embargo rules.

We consider this to be a sector problem – and one that should be addressed by sector-wide resources, such as the funding bodies. I should note Jisc have expressed interest in supporting us with this issue. The increased crack-down on our activities is frustrating, unhelpful and feeds the beast that is the publishing industry stranglehold on researcher behaviour.

Published 20 May 2016
Written by Dr Danny Kingsley
Creative Commons License

Open Data – moving science forward or a waste of money & time?

Uncategorized, , , , , , , , , , , ,

On the 4 November the Research Data Facility at Cambridge University invited some inspirational leaders in the area of research data management and asked them to address the question: “is open data moving science forward or a waste of money & time?”. Below are Dr Marta Teperek’s impressions from the event.

Great discussion

Want to initiate a thought-provoking discussion on a controversial subject? The recipe is simple: invite inspirational leaders, bright people with curious minds and have an excellent chair. The outcome is guaranteed.

We asked some truly inspirational leaders in data management and sharing to come to Cambridge to talk to the community about the pros and cons of data sharing. We were honoured to have with us:

  • PRE_IntroSlide_V3_20151123Rafael Carazo-Salas, Group Leader, Department of Genetics, University of Cambridge
    @RafaCarazoSalas
  • Sarah Jones, Senior Institutional Support Officer from the Digital Curation Centre; @sjDCC
  • Frances Rawle, Head of Corporate Governance and Policy, Medical Research Council; @The_MRC
  • Tim Smith, Group Leader, Collaboration and Information Services, CERN/Zenodo; @TimSmithCH
  • Peter Murray-Rust, Molecular Informatics, Dept. of Chemistry, University of Cambridge, ContentMine; @petermurrayrust

The discussion was chaired by Dr Danny Kingsley, the Head of Scholarly Communication at the University of Cambridge (@dannykay68).

What is the definition of Open Data?

IMG_PMRWithText_V1_20151126The discussion started off with a request for a definition of what “open” meant. Both Peter and Sarah explained that ‘open’ in science was not simply a piece of paper saying ‘this is open’. Peter said that ‘open’ meant free to use, free to re-use, and free to re-distribute without permission. Open data needs to be usable, it needs to be described, and to be interpretable. Finally, if data is not discoverable, it is of no use to anyone. Sarah added that sharing is about making data useful. Making it useful also involves the use of open formats, and implies describing the data. Context is necessary for the data to be of any value to others.

What are the benefits of Open Data?

IMG_RCSWithText_V1_20151126Next came a quick question from Danny: “What are the benefits of Open Data”? followed by an immediate riposte from Rafael: “What aren’t the benefits of Open Data?”. Rafael explained that open data led to transparency in research, re-usability of data, benchmarking, integration, new discoveries and, most importantly, sharing data kept it alive. If data was not shared and instead simply kept on the computer’s hard drive, no one would remember it months after the initial publication. Sharing is the only way in which data can be used, cited, and built upon years after the publication. Frances added that research data originating from publicly funded research was funded by tax payers. Therefore, the value of research data should be maximised. Data sharing is important for research integrity and reproducibility and for ensuring better quality of science. Sarah said that the biggest benefit of sharing data was the wealth of re-uses of research data, which often could not be imagined at the time of creation.

Finally, Tim concluded that sharing of research is what made the wheels of science turn. He inspired further discussions by strong statements: “Sharing is not an if, it is a must – science is about sharing, science is about collectively coming to truths that you can then build on. If you don’t share enough information so that people can validate and build up on your findings, then it basically isn’t science – it’s just beliefs and opinions.”

IMG_TSWithText_V1_20151126Tim also stressed that if open science became institutionalised, and mandated through policies and rules, it would take a very long time before individual researchers would fully embrace it and start sharing their research as the default position.

I personally strongly agree with Tim’s statement. Mandating sharing without providing the support for it will lead to a perception that sharing is yet another administrative burden, and researchers will adopt the ‘minimal compliance’ approach towards sharing. We often observe this attitude amongst EPSRC-funded researchers (EPSRC is one of the UK funders with the strictest policy for sharing of research data). Instead, institutions should provide infrastructure, services, support and encouragement for sharing.

Big data

Data sharing is not without problems. One of the biggest issues nowadays it the problem of sharing of big data. Rafael stressed that with big data, it was extremely expensive not only to share, but even to store the data long-term. He stated that the biggest bottleneck in progress was to bridge the gap between the capacity to generate the data, and the capacity to make it useful. Tim admitted that sharing of big data was indeed difficult at the moment, but that the need would certainly drive innovation. He recalled that in the past people did not think that one day it would be possible just to stream videos instead of buying DVDs. Nowadays technologies exist which allow millions of people to watch the webcast of a live match at the same time – the need developed the tools. More and more people are looking at new ways of chunking and parallelisation of data downloads. Additionally, there is a change in the way in which the analysis is done – more and more of it is done remotely on central servers, and this eliminates the technical barriers of access to data.

Personal/sensitive data

IMG_FRWithText_V1_20151126Frances mentioned that in the case of personal and sensitive data, sharing was not as simple as in basic sciences disciplines. Especially in medical research, it often required provision of controlled access to data. It was not only important who would get the data, but also what they would do with it. Frances agreed with Tim that perhaps what was needed is a paradigm shift – that questions should be sent to the data, and not the data sent to the questions.

Shades of grey: in-between “open” and “closed”

Both the audience and the panellists agreed that almost no data was completely “open” and almost no data was completely “shut”. Tim explained that anything that gets research data off the laptop to a shared environment, even if it was shared only with a certain group, was already a massive step forward. Tim said: “Open Data does not mean immediately open to the entire world – anything that makes it off from where it is now is an important step forward and people should not be discouraged from doing so, just because it does not tick all the other checkboxes.” And this is yet another point where I personally agreed with Tim that institutionalising data sharing and policing the process is not the way forward. To the contrary, researchers should be encouraged to make small steps at a time, with the hope that the collective move forward will help achieving a cultural change embraced by the community.

Open Data and the future of publishing

Another interesting topic of the discussion was the future of publishing. Rafael started explaining that the way traditional publishing works had to change, as data was not two-dimensional anymore and in the digital era it could no longer be shared on a piece of paper. Ideally, researchers should be allowed to continue re-analysing data underpinning figures in publications. Research data underpinning figures should be clickable, re-formattable and interoperable – alive.

IMG_DKWithText_V1_20151126Danny mentioned that the traditional way of rewarding researchers was based on publishing and on journal impact factors. She asked whether publishing data could help to start rewarding the process of generating data and making it available. Sarah suggested that rather than having the formal peer review of data, it would be better to have an evaluation structure based on the re-use of data – for example, valuing data which was downloadable, well-labelled, re-usable.

Incentives for sharing research data

IMG_SJWithText_V1_20151126The final discussion was around incentives for data sharing. Sarah was the first one to suggest that the most persuasive incentive for data sharing is seeing the data being re-used and getting credit for it. She also stated that there was also an important role for funders and institutions to incentivise data sharing. If funders/institutions wished to mandate sharing, they also needed to reward it. Funders could do so when assessing grant proposals; institutions could do it when looking at academic promotions.

Conclusions and outlooks on the future

This was an extremely thought-provoking and well-coordinated discussion. And maybe due to the fact that many of the questions asked remained unanswered, both the panellists and the attendees enjoyed a long networking session with wine and nibbles after the discussion.

From my personal perspective, as an ex-researcher in life sciences, the greatest benefit of open data is the potential to drive a cultural change in academia. The current academic career progression is almost solely based on the impact factor of publications. The ‘prestige’ of your publications determines whether you will get funding, whether you will get a position, whether you will be able to continue your career as a researcher. This, connected with a frequently broken peer-review process, leads to a lot of frustration among researchers. What if you are not from the world’s top university or from a famous research group? Will you be able to still publish your work in a high impact factor journal? What if somebody scooped you when you were about to publish results of your five years’ long study? Will you be able to find a new position? As Danny suggested during the discussion, if researchers start publishing their data in the ‘open”’ there is a chance that the whole process of doing valuable research, making it useful and available to others will be rewarded and recognised. This fits well with Sarah’s ideas about evaluation structure based on the re-use of research data. In fact, more and more researchers go to the ‘open’ and use blog posts and social media to talk about their research and to discuss the work of their peers. With the use of persistent links research data can be now easily cited, and impact can be built directly on data citation and re-use, but one could also imagine some sort of badges for sharing good research data, awarded directly by the users. Perhaps in 10 or 20 years’ time the whole evaluation process will be done online, directly by peers, and researchers will be valued for their true contributions to science.

And perhaps the most important message for me, this time as a person who supports research data management services at the University of Cambridge, is to help researchers to really embrace the open data agenda. At the moment, open data is too frequently perceived as a burden, which, as Tim suggested, is most likely due to imposed policies and institutionalisation of the agenda. Instead of a stick, which results in the minimal compliance attitude, researchers need to see the opportunities and benefits of open data to sign up for the agenda. Therefore, the Institution needs to provide support services to make data sharing easy, but it is the community itself that needs to drive the change to “open”. And the community needs to be willing and convinced to do so.

Further resources

  • Click here to see the full recording of the Open Data Panel Discussion.
  • And here you can find a storified version of the event prepared by Kennedy Ikpe from the Open Data Team.

Thank you

We also wanted to express a special ‘thank you’ note to Dan Crane from the Library at the Department of Engineering, who helped us with all the logistics for the event and who made it happen.

Published 27 November 2015
Written by Dr Marta Teperek
Creative Commons License