On 15th November 2017, the University of Cambridge held its first research data management (RDM) conference, Engaging Researchers in Good Data Management. The Office of Scholarly Communication collaborated with SPARC Europe and Jisc, hosted the one-day event at St. Catherine’s College. In attendance were researchers, administrators, and librarians all sharing their experiences with promoting good RDM. Having a mixture of people from various disciplines and backgrounds allowed many different points of view on engaging researchers to be discussed. In the afternoon, the attendees split off into focus groups to concentrate on a number of nagging questions.
Our group’s topic of discussion: How do we effectively measure cultural change in attitudes towards data management? Leading the discussion was Marta Teperek from Delft University of Technology. There was a mixture of around 30 librarians and researchers from all over the world discussing strategies for engaging with researchers.
How do we set about achieving ‘cultural change’?
Marta started the conversation off by asking what everyone present was already doing at their institutions to engage researchers. Many shared their experiences and some frustrations at pushing good data management habits. One person shared that at his university the initial push toward better data management was achieved by creating and delivering RDM workshops for PhDs and young researchers in the Digital Humanities. These students were already interested in digital preservation, so they were a keen audience. Targeting PhD students and early career researchers may be a more effective strategy because they could develop good data management habits early in their careers. The earlier the intervention, the easier it would (hopefully) be.
Overall, most agreed that directly speaking to researchers is more effective than having initiatives relayed from the top-down. Attendees perceived compliance as a driver rather than a useful stick to persuade researchers to take data management seriously. Even if only a few researchers turned up to data management events, it was still increasing exposure.
Some argued for a multi-prong strategy. Initiatives like the Data Stewards at Delft TU and the Data Champions at the University of Cambridge were perceived as good ways to reach out to researchers in their departments and provide more customized advice. At the same time, having expectations of good data management relayed from on high could help creating greater impetus.
What do we mean by ‘cultural change’?
Naturally, the conversation progressed to what the phrase ‘cultural change’ actually means. It was difficult to determine in 45 minutes what kind of ‘cultural change’ we wanted to see within our different institutions. We started by asking some questions. What were our goals? What would need to happen before we said yes, the culture is changing? Which really meant what do we measure to find evidence of cultural change? Is it better metadata, more awareness of copyright, researchers reaching out to us for help, or an increase in number of grants awarded that would signal an actual change? It would seem that there could be many definitions of ‘cultural change’, but the crucial takeaway is that it is essential to define what your parameters of cultural change will be in the planning stages of any RDM programme.
Where is the evidence?
The conversation progressed to how do we find and gather evidence. With all of the work being done by researchers, librarians, and administrators, how do we know what is actually effective? We cannot state that engaging with researchers (which can be time-consuming) is working without having actual evidence to confirm it. A number of different ideas were discussed, with the time when feedback was gathered being a particular point of variance.
Quantifiable information such as number of datasets deposited, number of datasets downloaded and re-used, and number of grants with a Data Management Plan could be collected. For example, the University of Illinois conducted a detailed analysis of 1,260 data management plans using a controlled vocabulary list and looked at possible correlations between solutions for data management listed in funded and unfunded proposals.
Another method of benchmarking included asking researchers to periodically complete short surveys on data management practice in order to measure any noticeable changes. In that way, an institution can assess whether their engagement strategies work and whether it achieves the desirable effects (improvement of data management practice). Delft, EPFL, Cambridge and Illinois collaborated on development of an agreed set of survey questions. Conducting this same survey across different institutions enables benchmarking and comparison of the different techniques and how effective they are in achieving cultural change in data management. In addition to this survey, the team also interviews some researchers in order to gather additional qualitative data and more detailed insights into data management practice. The hope is that carrying out these quantitative surveys and qualitative interviews periodically will correct for the potential problem of self-selecting participants.
In the future
Ultimately, it turned out that most of those attending the focus group discussion were already working actively to develop systems to measure impact and gather feedback. However, the possibility of carrying out long-term cross-institutional research that would allow comparisons between different data management programmes is very tantalising. The final takeaway from this focus group discussion was that the majority of those attending would be very keen to take part in such research, so watch this space!
Published 18 December 2017
Written by Katie Hughes and Lucy Welch
