Tag Archives: BBSRC

The purpose, practicalities, pitfalls and policies of managing and sharing data in the UK

October 20, 2015UncategorizedBBSRC, data journals, EPSRC, figshare, file naming protocol, funders, metadata, open data, policy, RCUK, repository, researcherOffice of Scholarly Communication

As part of the Office of Scholarly Communication Open Access Week celebrations, we are uploading a blog a day written by members of the team. Tuesday is a piece by Dr Danny Kingsley reflecting the talk she gave this morning to the Royal Society of Chemistry, Chemical Information and Computer Applications Group conference – Measurement, Information and Innovation: Digital Disruption in the Chemical Sciences.

The data policy landscape

The policy position on data management in the UK is driven on many levels. Many institutions now have policies – an example is the Cambridge University Research Data Management Policy Framework. Increasingly publishers such as PLOS are requiring that research published in their journals is accompanied by the data underpinning the research. Some journals, such as Nature’s Scientific Data are specifically data-only journals

There has been a country-wide movement towards opening up data. Consultation on the Draft Concordat on Open Research Data released by the RCUK ended on 28 September. Cambridge coordinated a joint response to the Concordat with several other universities.

However the real driver for action this year has been funder policies – specifically, the Engineering and Physical Sciences Research Council (EPSRC) which announced it was going to (and has begun) checking compliance as of 1 May 2015.

The devil is in the detail

While the Research Councils UK have RCUK Common Principles on Data Policy stating “Publicly funded research data are a public good (…), which should be made openly available with as few restrictions as possible”, these common principles are idiosyncratic when looked at from the individual council perspective, as the graphic on the second page of this document demonstrates.

There are variations on whether a data management plan is required, where the data should be stored, the level of support offered and even whether this can be funded through the grant (in most cases it can, but not all).

Places to share data

Open (cross-disciplinary) repositories

These include commercial options such as figshare which is owned by Digital Science who also produce Symplectic Elements research management systems and are is an offshoot company to Macmillian/Springer.

Open source solutions such as Zenodo, an open dependable home for the long-tail of science, enabling researchers to share and preserve any research outputs in any size, any format and from any science, developed by CERN.

Disciplinary repositories

There are a significant number of disciplinary specific data repositories. In many ways these are the most natural place for data as disciplinary experts can curate the data. For example the Natural Environment Research Council (NERC) runs seven repositories.

The first repository ever created (in 1991) was arXiv, holding e-prints in Physics, Mathematics, Computer Science, Quantitative Biology, Quantitative Finance and Statistics. The public functional genomics data repository is Gene Expression Omnibus . Social science data can be deposited in the UK Data Service. The Oxford Text Archive holds literary and linguistic texts for higher education.

Institutional options

Cambridge University is using its DSpace repository Apollo to store and share data. To date the largest dataset we have received is 28 GB – huge datasets need to sit externally to this repository. Not all institutions provide a data repository service. There are significant overheads associated with this activity both for the technology and the people to upload and curate the data.

Journals

Journals are increasingly requiring researchers to publish (or at least provide links to) their supporting data alongside their research articles. PLOS brought in their data sharing policy in December 2013.

There is also a growing selection of data only journals on the market, for example Nature’s Scientific Data. Others include Journal of Open Archaeology Data, Open Health Data, Journal of Open Psychology Data, Gigascience, Bodiversity Data Journal and Earth System Science Data.

Aggregating services

The Australian National Data Service has built Research Data Australia (RDA) which collates and displays information about datasets held all over the country, both open and closed. As of the 20 October, the RDA contains 115,823 datasets, of which 23,322 are open.

There appears to be little appetite yet for this sort of service in the UK, or at least for providing funds to create one.

What are we actually trying to achieve ?

Cambridge University has taken a proactive approach to the RCUK data sharing policies by inviting funders to come and discuss issues with the research community. We have written up and published these discussions as an ‘In conversation with..’ series on our Unlocking Research blog.

In addition to clarification on some aspects of the policies, one of the questions we are trying to answer is what is the actual goal of these policies? Ben Ryan the Senior Manager, Research Outcomes of the EPSRC clarified that researchers needed to share:

the data that underpins publications
the data that validates research findings
the data that is worth keeping

To summarise the philosophical goals of the EPSRC policy:

The default position is ‘data should be open’
Published research findings should be testable
Maximise the impact of publicly funded research
Maintain public trust in science and research
They are trying to create a new research culture

Researcher responses

While these might seem like lofty and even admirable goals, it does not mean that the academic response to being informed of their grant requirement for sharing data have been met with welcoming arms. Far from it in some cases.

A small selection of the responses we have received in our meetings with over 1500 researchers this year at Cambridge include:

What’s the minimum we can get away with?
This is crap
‘They’ are just doing this because ‘they’ can
But it will take a huge effort to get the data in a useable form
No-one will look at it
What a waste of time

This has prompted us to play a game we call ‘data excuse bingo’ at some of our Research Data Management Workshops – see slide 16.

We are trying to start at the end

The problem might be that everyone is fixated on sharing data at the end of the research process. However this is one of the lesser data management activities if data management begins at the beginning.

The second of our “In conversation…” series was with Michael Ball the Strategy and Policy Manager at the Biotechnical and Biological Sciences Research Council (BBSRC). Amongst a long discussion about what exactly constitutes ‘data’ in the biological sciences, Michael emphasised that disciplines themselves must establish ways of dealing with data. This is the beginning of an ongoing process.

He noted that researchers need to consider how to deal with data from the beginning of a research project. Researchers can ask for money to manage data in the grant application (something which is currently quite rare).

The practice of sharing data requires the data to be: Accessible, Intelligible, Assessable and Reusable. So how do we achieve that?

Basics of Research Data Management

Good data management includes very basic practices such as:

Writing a research data management plan at the beginning of the research process – identifying all of these issues
Using a file naming protocol (including version control)
Backing up work in several places
Identifying any data that might be politically, personally or commercially sensitive
Determining who owns what data
Ensuring data that is being used for research across collaborations is shared in safe, secure and legal shared facilities, bearing in mind Export Control Legislation.
Having good metadata protocols
Using a reputable and reliable storage/sharing facility that offers persistent identifiers (DOIs)

Who owns the data?

This is an interesting question. EPSRC policies state that researchers should ensure that collaborators are aware of the sharing requirement before they embark on research. Then there are questions about who within the team might own the data – with again a suggestion to come to some sort of agreement before work starts.

Are all collaborators equal ‘owners’? Or does the Principal Investigator have a 50% stake, with the remaining split amongst the PhD students making up the remainder of the team? You might want to talk to your legal advisors and/or your research office about this issue.

There is also the related issue here of developing author-contribution transparency. Do you include author contribution statements in your articles?.

Staffing issues

If Michael Ball is correct and very few researchers are asking for funds associated with managing data, it is reasonable to assume that data is being managed in an ad hoc way – with reliance on the computer savvy postdoc the project hired …

Required skillsets for managing and curating data

There is a considerable range of skill sets associated with managing data, and these have been described by the Digital Curation Centre as data creator, data scientist, data librarian, data manager.

Alma Swan and Sheridan Brown’s 2008 report to Jisc on ‘The skills, role and career structure of data scientists and curators: an assessment of current practice and future needs’ described these as:

Data Creator: Researchers with domain expertise who produce data. These people may have a high level of expertise in handling, manipulating and using data
Data Scientist: People who work where the research is carried out – or, in the case of data centre personnel, in close collaboration with the creators of the data – and may be involved in creative enquiry and analysis, enabling others to work with digital data, and developments in data base technology
Data Manager: Computer scientists, information technologists or information scientists and who take responsibility for computing facilities, storage, continuing access and preservation of data
Data Librarian: People originating from the library community, trained and specialising in the curation, preservation and archiving of data

There is a simple graphic that clearly shows how these roles relate to one another.

Certainly an increasing number of data scientist jobs are being advertised. A search for ‘Data scientist’ + ‘London’ on the job site Indeed on 18 October produced 1,405 results. So where are all of these people coming from?

Training available

The Swan and Brown study in 2008 noted that ‘People in data science roles face a big, continuing challenge in remaining properly skilled up’ and this remain the situation today, although there are now many more opportunities for training – a few are listed here:

The Digital Curation Centre offers data management and curation education and training.
MANTRA is a free online course from the Data Library staff in Information Services, University of Edinburgh. It is designed for those who manage digital data as part of a research project. It was crafted for the use of post-graduate students, early career researchers, and also information professionals. It is freely available on the web for anyone to explore on their own.
Data Scientist training for Librarians is a collection of notes and discussions about data work done by librarians. They ran an experimental course in August to teach “the latest tools for extracting, wrangling, storing, analysing, and visualising data”.

In addition, the professionalisation of these skills is beginning. For example, Data Science London is a community of data scientists that meets regularly to discuss data science ideas, concepts, and tools, methods and technologies used by many startups to analyse large scale data (big data), extract predictive insight, and exploit business opportunities from data products. Their website offers a list of free data science courses.

Cambridge University is one of the five founding university partners of the about to be launched Alan Turing Institute, which is intended to be the UK’s national institute for data science. The Institute will be addressing some of the issues with the data management skill gap.

Issues with sharing data

For all of the ‘feel good’ ideals about sharing data, and the processes being put in place to support this, there are some serious issues raised by researchers about the requirement to share data.

To start, there is a very real concern that the UK will become unattractive for collaborations. Why would a commercial collaborator choose a UK partner when a partner from elsewhere is not under obligation to share their data?

There have been some discussions at information sessions about the possible need to change the type of research being done to reduce the amount of data being produced because of the cost of long term storage of this data.

Indeed, there is discussion in some circles about whether applying for EPSRC funding is worth the hassle. It is a fair bet that none of these were intended outcomes of the RCUK policies.

Consequences of not sharing data

However, this does need to be a balanced strategy. There is a considerable argument that openness is related to academic integrity as it allows work to be verified and validated.

Here are examples in three disciplines where sharing data had a dramatic effect:

Medicine – having the data publicly available in two trials of deworming pills demonstrated that a population wide deworming program did not improve school performance.
Economics – A study widely cited to justify budget cutting in the US had a mistake in the calculations which was only revealed when the Excel file was released
Physics – It took 12.5 years to withdraw Jan Hendrik Schon’s work on ‘organic semiconductors’ because the reviewers were unable to replicate the results without access to the original data or lab books.

Conclusion

Sharing data offers great challenges to the research community, not least because it is less than clear what ‘data’ means in different disciplines. It will take some time for the research community to change its philosophy and practice. But the positives outweigh the negatives and with hope we will look back at this time as a short transition period.

Note the slides from the talk are available in Slideshare.

Published 20 October 2015
Written by Dr Danny Kingsley

In conversation with Michael Ball from BBSRC

October 19, 2015UncategorizedBBSRC, biological sciences, funders, open data, policyOffice of Scholarly Communication

The Biotechnical and Biological Sciences Research Council (BBSRC) Data Sharing Policy states that research data that supports publications must be stored for 10 years and adherence to data management plans will be monitored and built into the Final Report score, which may be taken into account for future proposals.

Recently Michael Ball, the Strategy and Policy Manager at BBSRC accepted an invitation to Cambridge University to discuss the BBSRC policy on opening up access to data. Senior members of the University, the School of Biological Sciences, the Research Office and the Office of Scholarly Communications attended. These notes have been verified by Michael as an accurate reflection of the discussion.

The take home messages from the meeting were the importance of:

Disciplines themselves establishing ways of dealing with data
Thinking about how to deal with data from the beginning of a research project

The meeting began with a discussion about the support we provide Cambridge University researchers through the Research Data Service , the resources provided on the data website and the enthusiastic uptake of the service since the beginning of the year.

The conversation then moved into issues around the policy, focusing on several aspects – clarification of what needs to be shared, how this will be supported financially, questions about auditing, a discussion about the best place to keep the data and issues with data sharing in the biological sciences.

What data are we expected to share?

What is ‘supporting data’ in the biological sciences?

One of the biggest concerns biological researchers have about data sharing is what is meant by ‘data’. Biology has the most diverse group of data, which makes it hard to talk about biology because the issues are project and problem specific.

Michael confirmed the policy broadly refers to all data ‘but the devil is in the detail, there are lots of caveats’. He echoed Ben Ryan in answer to a similar question of the EPSRC policy by saying the key points are:

What would you expect to see?
What do you think is important?

The interpretation of the BBSRC policy depends heavily on the types of data being produced. Much is dependent on the expected norms, what a researcher would expect to see if they were trying to interpret the paper. What are the underlying supporting data for the paper?

The biological sciences throw up a particular challenge in the range and disparity in disciplinary norms. For example a great deal of data arises from genomics and some time ago they made the decision to share, including making decisions about what to share and what not to share. However, there are vast areas of experimental science where the paper itself is data.

The policy is going one step further back from the published paper towards the lab. In the future these data policies might go further back, if there was greater automation of the research process.

Michael confirmed that if the BBSRC has funded a PhD student they would expect them to make supporting data available.

What do we need to share in the Biological Sciences?

There is no expectation to share lab books unless they are the only place the data exists. Michael noted that when the BBSRC wrote the policy it excluded lab books and organisms.

However there is an expectation to share instrumental output. This is with the caveat that if it is output from an instrument that goes through some sort of amendment then you don’t need to share the original.

An example: A researcher is counting bacteria on a plate and scrupulously making notes in lab books before entering this information put into a computer spreadsheet to crunch the numbers. The expectation would be to share the spreadsheet not the lab book.

Some research requires the construction of a piece of technology where there might not be a great deal of associated data around it. In these instances it is the process of construction or the protocol or the methodology that is important to share.

Michael noted that in some disciplines, given the materials and input parameters and the same instruments, the output data will be the same each time. In these circumstances it is most sensible to share or describe the inputs and repeat the experiments. The question is about what would be the most useful to share.

Show me the money

A stitch in time

Michael confirmed that researchers can ask for the money they need (and can justify) for research data management in grant applications. He did say however that the BBSCR does not ‘generally see a lot of these requests’. He noted that this is because often people haven’t thought about the data they will generate at the start of the project. One of the researchers pointed out it was difficult to know how to fund it because ‘we are not sure what we need’. However, this should not be a reason to ask for nothing.

It may be that some of the discipline specific repositories will have to change their business models in the future to cope with larger data sets.

Michael said that it is worth thinking about data sharing at the project planning stage because different types of data have different requirements. Researchers might need to allow for the cost of getting the data in the right format and metadata. It is advisable to think about where the data will be published so the research team can prepare the data in the first instance.

Michael said that the data management plan should hopefully prompt how much data a research project will produce. It is advisable to consider the maximum amount of data the project may produce. The ideal situation will be to have an ongoing data management plan because in some ways it is useful at the end.

Longer term financial support

Raised in the meeting was the option of charging a flat fee up front regardless of the data being generated. The question arose about whether there was any danger in auditing with this approach? The problem with an up front fee is it becomes more difficult to track and output from a specific grant against what we put into the database. There is a directly incurred and directly allocated component to the cost.

Michael confirmed that any money allocated to data management won’t survive past the end of the grant. He noted this was something that he was ‘not sure how to unpick’. This raises the issue of the cost of longer term data sharing. The BBSRC provides funding to a certain point in time. There can be a secondary experiment funded by someone else and the works are published together. But the researcher can only share the data from the funded part. The BBSRC does not ask researchers to share data that they haven’t funded.

Auditing questions

Who is in charge here?

The academics raised the concern that there could be ‘mission creep’ where the funders expect people to do things that are a waste of time. They mentioned that an ideal situation would be where the research community decide what they want to share and what they don’t wish to share.

Michael noted that the BBSRC has to be guided by the community on their own community norms for data sharing, and this is why aspects of the data sharing policy is quite open. He noted that this meeting represented the first part of the process – where the funder comes together with communities to decide what is essential.

In addition, many journals are now requiring open data. It is the funders, the researchers and the journals who are asking for it. To some extent the BBSRC policy is guided by what the journals are asking for.

The policing process

The group expressed interest in how the BBSRC policy is policed and what would be the focus of that policing. Michael stated that BBSRC are investigating options of how to monitor compliance, but that it does not currently appear feasible to to check all of the submissions. BBSRC will monitor compliance, but will probably start with dipstick testing. They will look at historical projects and see where the process goes from there. In practice, this is likely to initially involve examining the degree of adherence to the submitted data management plans. If a researcher has acted reasonably and justified their mechanisms of data sharing, then it is unlikely that there would be any actions beyond noting where difficulties had occurred.

Note, however that if a researcher has submitted a grant application with a data sharing statement there is a reasonable expectation to share the data.

Ultimately the data release will be policed. In areas where data sharing is prevalent, communities police themselves because researchers ask and expect the data to be available. In some cases you can’t publish without an accession number.

Michael noted there are places researchers can put information about published data into ResearchFish. ResearchFish is currently the only mechanism to capture information regarding post-award activities.

Where do we put the data?

The question arose about how other universities are managing the policy. Michael responded that many have started institutional repositories. The institutional response depends on where the majority of their research sits.

A possible solution for ensuring the data is discoverable would be a catalogue of what is stored in an institutional repository, with metadata about the data. That metadata would itself need to be discoverable. If the data is being held in a centralised repository it is possible to pay the cost upfront before the end of the grant.

The group noted there was a publishing preference for discipline specific repositories over institutional repositories because the community knows how to look after the work. These repositories are hosted by ‘people who know what they are doing’. They are discoverable, where the community can decide on the metadata and the required standards.

Michael agreed that the ideal was open discoverability. The question is what will be practically possible.

A way of considering the question is asking how would another researcher find the information? If the data is available from a researcher by request this should be noted in the paper. If it is available in a repository then the paper should state that. If the journal has told readers where the data is, then it should be self-evident.

Issues with obsolescence

Michael noted that there is an ongoing issue of obsolete data formats and disks. Given there are ideals and reality, it becomes a question of how to store and handle the information.

When data exists in a proprietary format, the researcher needs to think about how to access it in the longer term. What if the organisation goes out of business? Or the technology upgrades so you can’t get hold of the data in an earlier format? If data exists in a physical format then it is possible to go back and read it. However, if not then it is quite important to think about issues relating to long-term access. Lots of data will be obsolete.

There are some solutions for this issue. The Open Microscopy Environment is a joint project between universities, research establishments, industry and the software development community. It develops open-source software and data format standards for the storage and manipulation of biological microscopy data. This is a community-generated solution as a recognised problem. It has a database that you can upload any file format.

Issues with data sharing in the biological sciences

The BBSRC allows a reasonable embargo until the researcher has exploited the data for publication. If the researcher is planning on releasing further publications then they should consider carefully when to release the data., Michael noted, this is ‘not a forever thing’. The BBSRC do say there are reasonable limits, and some journals will expect data to be released alongside publications.

Commercial partners

Data emerging from BBSRC funded research needs to be shared unless there is a reason why not – and commercial partners who need to protect their intellectual property can be a good reason to delay data sharing. However once the Intellectual Property is protected, it is protected. The BBSRC allows researchers to embargo the data.

Michael also noted there are things that can be done with data, for example releasing it under license. An example is, if a researcher is working with a commercial partner who is concerned about other commercial competitors, it would be possible to require people to sign non-disclosure agreements. There are ways to deal with commercial data, as you would with other intellectual products.

It was noted by the researchers in the meeting that this type of arrangement is likely to mean the company doesn’t want to go through the process and won’t collaborate.

Exceptions

If data was generated before the policy was in place then the researcher has not submitted a grant application that requires them to share their data. The BBSRC is not expecting people to go back into history. Those researchers who wish to share historical research are not discouraged but this is not covered by the policy. The policy came into force in April 2007, however realistically it started in 2008.

In addition there are reasonable grounds for not sharing clearly incorrect or poor quality data. Many disciplinary databases will contain an element of quality control. But Michael noted that the policy shouldn’t be a way for people to filter out inconvenient data and would expect the community to be self policing.

Future policy direction

Michael noted that this type of policy is becoming more prevalent not less. Open science is one of the Horizon 2020 themes – see the 2013 Guidelines on Open Access to Scientific Publications and Research Data in Horizon 2020. Journals are getting involved as well. In the future sharing data will be more common – and driven by disciplinary norms. Anything that has been funded by RCUK will be required to share. It makes sense to government – the US National Institutes of Health and National Science Foundation have data sharing statements.

Continuing the dialogue

Michael indicated that he wants to talk to people about what the questions are so the BBSRC can refine issues in the policy.

Researchers who have questions about the policy can send them through to the Research Data Service team info@data.cam.ac.uk. If we are unable to answer them, we can ask BBSRC directly for clarification. We will then add the information to the University Research Data Management FAQ webpage.

Published 19 October 2015
Written by Dr Danny Kingsley, verified by Michael Ball, BBSRC