Tag Archives: data

Lessons learned from Jisc Research Data Champions

Uncategorized, ,

In 2017 four Cambridge researchers received grants from Jisc to develop and share their research data management practices. In this blog, the four awardees each highlight one aspect of their work as a Jisc Data Champion.

The project

All four Champions embarked on a range of activities throughout the year including creating local communities interested in RDM practices, delivering training, running surveys to understand their department better, creating ‘how-to’ guides for would-be RDM mentors and testing Samvera as part of RDSS. They were excited by the freedom that the grant gave them to try out whatever RDM related activities they wanted, which meant they could develop their skills and see ideas come to fruition and make them reusable for others. For example, Annemarie Eckes developed a questionnaire on RDM practices for PhD students and Sergio Martínez Cuesta has posted his training courses on GitHub.  

However, throughout the duration of the award they also found some aspects of championing good RDM disconcerting. Whilst some sessions proved popular, others had very low attendee figures, even when a previous iteration of the session was well attended. They all shared the sense of frustration often felt by central RDM services that it is getting people to initially engage and turn up to a session that is the hard part. However, when people did come they found the sessions very useful, particularly because the Champions were able to tailor it specifically to the audience and discipline and the similar background of all the attendees provided an extra opportunity for exchanging advice and ideas that were most relevant.

The Champions tried out many different things. The Jisc Research Data Champions were expected to document and publicise their research data management (RDM) experiences and practices and contribute to the Jisc Research Data Shared Service (RDSS) development. Here the Champions each highlight one thing they tried out, which we hope will help others with their RDM engagement.

BYOD (Bring your own data)

Champion: Annemarie Eckes, PhD student, Department of Geography

The “Bring your own data” workshop was intended for anyone who thought their project data needed sorting, they needed better documentation, or even they needed to find out who is in charge or the owner of certain data. I set it up to give attendees time and space to do any kind of data-management related tasks: clean up their data, tidy up their computer/ email inbox, etc. The workshop was, really, for everyone whether at the start of their project and at the planning stage or in the middle of a project and had neglected their data management to some extent.

For the workshop the participants needed a laptop or login for the local computers to access their data and a project to tidy up or prepare, that can be done within two hours. I provided examples of file naming conventions and folder structures as well as instructions on how to write good READMEs (messages to your future self) and a data audit framework to give participants some structure to their organisation. After a brief introductory presentation about the aims and the example materials I provided, people would spend the rest of the time tidying up their data or in discussions with the other participants.

While this was an opportunity for the participants to sit down and sort out their digital files, I also wanted participants to talk to each other about their data organisation issues and data exchange solutions. Once I got everyone talking, we soon discovered that we have similar issues and were able to exchange information on very specific solutions.

1-on-1 RDM Mentoring

Champion: Andrew Thwaites, postdoc, Department of Psychology

I decided to trial 1-on-1 RDM mentoring as a way to customise RDM support for individual researchers in my department. The aim was that by the end of the 1-on-1 session, the mentee should understand how to a) share their data appropriately at the end of their project, and b) improve on their day-to-day research data management practice.

Before the meeting, I encouraged the mentee to compile a list of funders, and their funder’s data sharing requirements. During the meeting, the mentee and I would make a list of the data in the mentees project that they are aiming to share, and then I would then help them to choose a repository (or multiple repositories) to share this data on, and I’d also assist in designing the supporting documentation to accompany it. During the sessions I also had conversations about about GDPR, anonymising data, internal documentation and day-to-day practices (file naming conventions, file backups etc.) with the mentee.

As far as possible, I provided non-prescriptive advice, with the aim being to help the mentee make an informed decision, rather than forcing them into doing what I thought was best.

Embedding RDM  

Champion: Sergio Martinez Cuesta, research associate, CRUK-CI and Department of Chemistry

I came to realise early in the Jisc project that stand-alone training sessions focused exclusively on RDM concepts were not successful as students and researchers found them too abstract, uninteresting or detached from their day-to-day research or learning activities. I think the aerial view of the concept of 1-on-1 mentoring and BYOD sessions is beautiful. However, in my opinion, both strategies may face challenges with necessary numbers of mentors/trainers increasing unsustainably as the amount of researchers needing assistance grows and the research background of the audience becomes more diverse.

To facilitate take-up, I tapped into the University’s lists of oversubscribed computational courses and found that many researchers and students already shared interests in learning programming languages, data analysis skills and visualisation in Python and R. I explored how best to modify some of the already-available courses with an aim of extending the offer after having added some RDM concepts to them. The new courses were prepared and delivered during 2017-2018. Some of the observations I made were:

  • Learning programming naturally begs for proper data management as research datasets and tables need to be constantly accessed and newly created. It was helpful to embed RDM concepts (e.g. appropriate file naming and directory structure) just before showing students how to open files within a programming language.
  • The training of version control using git required separate sessions. Here students and researchers also discover how to use GitHub, which later helps them to make their code and analyses more reproducible, create their own personal research websites …
  • Gaining confidence in programming, structuring data / directories and version control in general helps students to acknowledge that research is more robust when open and contrasted by other researchers. Learning how researchers can identify themselves in a connected world with initiatives such as ORCID was also useful.

Brown Bag Lunch Seminar Series: The Productive Researcher

Champion: Melissa Scarpate, postdoc, Faculty of Education

I created the Productive Researcher seminar series to provide data management and Open Access information and resources to researchers at the Faculty of Education (FoE). The aim of the brown bag lunch format was to create an informal session where questions, answers and time for discussion could be incorporated. I structured the seminars so they covered 1) a presentation and discussion of data management and storage; 2) a presentation about Open Access journals and writing publications; 3) a presentation on grant writing where Open Access was highlighted.

While the format of the series was designed to increase attendance, the average was four attendees per session. The majority of attendees were doctoral students and postdocs who had a keen interest in properly managing their data for their theses or projects. However, I suspect it may be the case that those attending already understood data management processes and resources.

In conclusion, I think that whilst the individuals that attended these seminars found the content helpful (per their feedback) the impact of the seminars was extremely limited. Therefore, my recommendation would be to have all doctoral students take a mandatory training class on data management and Open Access topics as part of their methodological training. Furthermore, I think it may be most helpful in reaching postdocs and more senior researchers to have a mandated data management meetings with a data manager to discuss their data management and Open Access plans prior to submitting any grant proposals. Due to new laws and policies on data (GDPR) this seems a necessary step to ensure compliance and excellence in research.

Published 2 October 2018
Compiled and edited by Dr Lauren Cadwallader from contributions by Annemarie Eckes, Dr Andrew Thwaites, Dr Sergio Martinez Cuesta, Dr Melissa Scarpate
Creative Commons License

Benchmarking RDM Training

Uncategorized, , ,

This blog reports on the progress of the international project to benchmark Research Data Management training across institutions. It is a collaboration of Cambridge Research Data Facility staff with international colleagues – a full list is at the bottom of the post. This is a reblog, the original appeared on 6 October 2017. 

How effective is your RDM training?

When developing new training programmes, one often asks oneself a question about the quality of training. Is it good? How good is it? Trainers often develop feedback questionnaires and ask participants to evaluate their training. However, feedback gathered from participants attending courses does not answer the question how good was this training compared with other training on similar topics available elsewhere. As a result, improvement and innovation becomes difficult. So how to objectively assess the quality of training?

In this blog post we describe how, by working collaboratively, we created tools for objective assessment of RDM training quality.

Crowdsourcing

In order to objectively assess something, objective measures need to exist. Being unaware of any objective measures for benchmarking of a training programme, we asked Jisc’s Research Data Management mailing list for help. It turned out that a lot of resources with useful advice and guidance on creation of informative feedback forms was readily available, and we gathered all information received in a single document. However, none of the answers received provided us with the information we were looking for. To the contrary, several people said they would be interested in such metrics. This meant that objective metrics to address the quality of RDM training either did not exist, or the community was not aware of them. Therefore, we decided to create RDM training evaluation metrics.

Cross-institutional and cross-national collaboration

For metrics to be objective, and to allow benchmarking and comparisons of various RDM courses, they need to be developed collaboratively by a community who would be willing to use them. Therefore, the next question we asked Jisc’s Research Data Management mailing list was whether people would be willing to work together to develop and agree on a joint set of RDM training assessment metrics and a system, which would allow cross-comparisons and training improvements. Thankfully, the RDM community tends to be very collaborative, which was the case also this time – more than 40 people were willing to take part in this exercise and a dedicated mailing list was created to facilitate collaborative working.

Agreeing on the objectives

To ensure effective working, we first needed to agree on common goals and objectives. We agreed that the purpose of creating the minimal set of questions for benchmarking is to identify what works best for RDM training. We worked with the idea that this was for ‘basic’ face-to-face RDM training for researchers or support staff but it can be extended to other types and formats of training session. We reasoned that same set of questions used in feedback forms across institutions, combined with sharing of training materials and contextual information about sessions, should facilitate exchange of good practice and ideas. As an end result, this should allow constant improvement and innovation in RDM training. We therefore had joint objectives, but how to achieve this in practice?

Methodology

Deciding on common questions to be asked in RDM training feedback forms

In order to establish joint metrics, we first had to decide on a joint set of questions that we would all agree to use in our participant feedback forms. To do this we organised a joint catch up call during which we discussed the various questions we were asking in our feedback forms and why we thought these were important and should be mandatory in the agreed metrics. There was lots of good ideas and valuable suggestions. However, by the end of the call and after eliminating all the non-mandatory questions, we ended up with a list of thirteen questions, which we thought were all important. These however were too many to be asked of participants to fill in, especially as many institutions would need to add their own institution-specific feedback questions.

In order to bring down the number of questions which should be made mandatory in feedback forms, a short survey was created and sent to all collaborators, asking respondents to judge how important each question was (scale 1-5, 1 being ‘not important at all that this question is mandatory’ and 5 being ‘this should definitely be mandatory’.). Twenty people participated in the survey. The total score received from all respondents for each question were calculated. Subsequently, top six questions with the highest scores were selected to be made mandatory.

Ways of sharing responses and training materials

We next had to decide on the way in which we would share feedback responses from our courses and training materials themselves . We unanimously decided that Open Science Framework (OSF) supports the goals of openness, transparency and sharing, allows collaborative working and therefore is a good place to go. We therefore created a dedicated space for the project on the OSF, with separate components with the joint resources developed, a component for sharing training materials and a component for sharing anonymised feedback responses.

Next steps

With the benchmarking questions agreed and with the space created for sharing anonymised feedback and training materials, we were ready to start collecting first feedback for the collective training assessment. We also thought that this was also a good opportunity to re-iterate our short-, mid- and long-term goals.

Short-term goals

Our short-term goal is to revise our existing training materials to incorporate the agreed feedback questions into RDM training courses starting in the Autumn 2017. This would allow us to obtain the first comparative metrics at the beginning of 2018 and would allow us to evaluate if our designed methodology and tools are working and if they are fit for purpose. This would also allow us to iterate over our materials and methods as needed.

Mid-term goals

Our mid-term goal is to see if the metrics, combined with shared training materials, could allow us to identify parts of RDM training that work best and to collectively improve the quality of our training as a whole. This should be possible in mid/late-2018, allowing time to adapt training materials as result of comparative feedback gathered at the beginning of 2018 and assessing whether training adaptation resulted in better participant feedback.

Long-term goals

Our long-term goal is to collaboratively investigate and develop metrics which could allow us to measure and monitor long-term effects of our training. Feedback forms and satisfaction surveys immediately after training are useful and help to assess the overall quality of sessions delivered. However, the ultimate goal of any RDM training should be the improvement of researchers’ day to day RDM practice. Is our training really having any effects on this? In order to assess this, different kinds of metrics are needed, which would need to be coupled with long-term follow up with participants. We decided that any ideas developed on how to best address this will be also gathered in the OSF and we have created a dedicated space for the work in progress.

Reflections

When reflecting on the work we did together, we all agreed that we were quite efficient. We started in June 2017, and it took us two joint catch up calls and a couple of email exchanges to develop and agree on joint metrics for assessment of RDM training. Time will show whether the resources we create will help us meet our goals, but we all thought that during the process we have already learned a lot from each other by sharing good practice and experience. Collaboration turned out to be an excellent solution for us. Likewise, our discussions are open to everyone to join, so if you are reading this blog post and would like to collaborate with us (or to follow our conversations), simply sign up to the mailing list.

Resources

Published 9 October 2017
Written by: (in alphabetical order by surname): Cadwallader Lauren, Higman Rosie, Lawler Heather, Neish Peter, Peters Wayne, Schwamm Hardy, Teperek Marta, Verbakel Ellen, Williamson, Laurian, Busse-Wicher Marta
Creative Commons License

Milestone -1000 datasets in Cambridge’s repository

Uncategorized, , , ,

Last week, Cambridge celebrated a huge milestone – the deposit of the 1000th dataset to our repository Apollo since the launch of the Research Data Facility in early 2015. This is the culmination of a huge amount of work by the team in the Office of Scholarly Communication, in terms of developing systems, workflows, policies and through an extensive advocacy campaign. The Research Data team have run 118 events over the past couple of years and published 39 blogs.

In the past 12 months alone there have been 26000 downloads of the data in Apollo. In some cases the dataset has been downloaded many times – 170 – and the data has featured in news, blogs and Twitter.

An event was held at Cambridge University Library last week to celebrate this milestone.

   

Opening remarks

The Director of Library Services, Dr Jess Gardner opened proceedings with a speech where she noted “the Research Data Services and all who sail in her are at the core of our mission in our research library”.

Dr Gardner referred to the library’s long and proud history of collecting and managing research data that “began on vellum, paper, stone and bone”. The research data of luminaries such as Isaac Newton and Charles Darwin was on paper and, she noted “we have preserved that with great care and share it openly on line through our digital library.”

Turning to the future, Dr Gardner observed: “But our responsibility now is today’s researcher and today’s scientists and people working across all disciplines across our great university. Our preservation stewardship of that research data from the digital humanities across the biomedical is a core part of what we now do.”

“In the 21st century our support and our overriding philosophy is all about supporting open research and opening data as widely as possible,” she noted.  “It is about sharing freely wherever it is appropriate to do so”. [Dr Gardner’s speech is in full at the end of this post.]

Perspectives from a researcher

The second speaker was Zoe Adams, a PhD student at Cambridge who talked about the work she has done with Professor Simon Deakin on the Labour Regulation Index in association with the Centre for Business Research.

Ms Adams noted it was only in retrospect she could “appreciate the benefit of working in a collaborative project and open research generally”. She discussed how helpful it had been as an early career researcher to be “associated with something that was freely available”. She observed that few of her peers had many citations, and the reason she did was because “the dataset is online, people use the data, they cite the data, and cite me”.

Working openly has also improved the way she works, she explained, saying “It has given me a new perspective on what research should be about. …  It gives me a sense that people are relying on this data to be accurate and that does change the way you approach it.”

View from the team

The final speaker was Dr Lauren Cadwallader, Joint Deputy Head of the OSC with responsibility for the Research Data Facility, who discussed the “showcase dataset of the data that we can produce in the OSC” which is  taken from usage of our Request a Copy service.

Dr Cadwallader noted there has been an increase in the requests for theses over time. “This is a really exciting observation because the Board of Graduate studies have agreed that all students should deposit a digital copy of their thesis in our repository,” she said. “So it is really nice evidence that we can show our PhD students that by putting a copy in the repository people can read it and people do want to read theses in our repository.”

One observation was that several of the theses that were requested were written 60 years ago, so the repository is sharing older research as well. The topics of these theses covered algebra, Yorkshire evangelists and one of the oldest requested theses was written in 1927 about the Falkland Islands. “So there is a longevity in research and we have a duty to provide access to that research, ” she said.

Thanks go to…

The dataset itself is one created by the OSC team looking at the usage of our Request a Copy service. The analysis undertaken by Peter Sutton Long and we recently published a blog post about the findings.

The music played at the event was complied by Tony Malone and covers almost 1000 years of music, from Laura Cannell’s reworking of Hildegard of Bingen, to Jane Weaver’s Modern Cosmology. There are acknowledgments to Apollo, and Cambridge too. The soundtrack is available for those interested in listening.

This achievement is entirely due to the incredible work of the team in the Research Data Facility and their ability to engage with colleagues across the institution, the nation and the world. In particular the vision and dedication of Dr Marta Teperek cannot be understated.

In the words of Dr Gardner: “They have made our mission different, they have made our mission better, through the work they have achieved and the commitment they have.”

The event was supported by the Arcadia Fund, a charitable fund of Lisbet Rausing and Peter Baldwin.

 

 

Published 21 September 2017
Written by Dr Danny Kingsley
Creative Commons License

Speech by Dr Jess Gardner

First let us begin with some headline numbers. One thousand datasets. This is hugely significant and a very high level when looking at research repositories around the country. There is every reason to be proud of that achievement and what it means for open research.

There have been 26000 downloads of that data in the past 12 months alone – that is about use and reuse of our research data and is changing the face of how we do research. Some of these datasets have been downloaded 117 times and used in news, blogs and Twitter. The Research Data team have written 39 blogs about research data and have run 118 events, most of these have been with researchers.

While the headline numbers give us a sense of volume, perhaps let’s talk about the underlying rationale and philosophy behind this, which is core.

Cambridge University Library has a 600 year old history we are very proud of. In that time we have had an abiding responsibility to collect, care for and make available for use and reuse, information and research objects that form part of the intrinsic international scholarly record of which Cambridge has been such a strong part. And the ability for those ideas to inspire new ideas. The collection began on vellum, paper, and stone and bone.

And today much of that of course is digital. You can’t see that in the same way you can see the manuscripts and collections. It is sometimes hard to grasp when we are in this grand old dame of a building that I dare you not to love. It is home to the physical papers of such greats as Isaac Newton and Charles Darwin. Their research data was on paper and we have preserved that with great care and share it openly on line through our digital library. But our responsibility now is today’s researcher and today’s scientists and people working across all disciplines across our great university. Our preservation stewardship of that research data from the digital humanities across the biomedical is a core part of what we now do.

And the people in this room have changed that. They have made our mission different, they have made our mission better through the work they have achieved and the commitment they have.

Philosophically this is very natural extension of what we have done in the Library and the open library and its great research community for which this very building is designed. Some of you may know there is a philosophy behind this building and the famous ‘open library Cambridge’. In the 19th century and 20th century that was mostly about our open stack of books and we have quite a few of them, we are a little weighed down by them.

Our research data weighs less but it is just as significant and in the 21st century our support and our overriding philosophy is all about supporting open research and opening data as widely as possible. It is about sharing freely wherever it is appropriate to do so and there are many reasons why data isn’t open sometimes, and that is fine. What we are looking for is managing so we can make those choices appropriately, just as we have with the archive for many, many years.

So whilst as there is a fantastic achievement to mark tonight with those 1000 datasets it really is significant, we are really celebrating a deeper milestone with our research partners, our data champions, our colleagues in the research office and in the libraries across Cambridge, and that is about the changing role in research support and library research support in the digital age, and I think that is something we should be very proud of in terms of what we have achieved at Cambridge. I certainly am.

I am relatively new here at Cambridge. One of the things that was said to me when I was first appointed to the job was how lucky I was to be working at this University but also with the Office of Scholarly Communication in particular and that has proved to be absolutely true. I like to take this opportunity to note that achievement of 1000 datasets and to state very publicly that the Research Data Services and all who sail in her are at the core of our mission in our research library. But also to thank you and the teams involved for your superb achievements. It really is something to be very proud of and I thank you.