Tag Archives: Jisc

How do you know if you’re achieving cultural change?

On 15th November 2017, the University of Cambridge held its first research data management (RDM) conference, Engaging Researchers in Good Data Management. The Office of Scholarly Communication collaborated with SPARC Europe and Jisc, hosted the one-day event at St. Catherine’s College. In attendance were researchers, administrators, and librarians all sharing their experiences with promoting good RDM. Having a mixture of people from various disciplines and backgrounds allowed many different points of view on engaging researchers to be discussed. In the afternoon, the attendees split off into focus groups to concentrate on a number of nagging questions.

Our group’s topic of discussion: How do we effectively measure cultural change in attitudes towards data management? Leading the discussion was Marta Teperek from Delft University of Technology. There was a mixture of around 30 librarians and researchers from all over the world discussing strategies for engaging with researchers.

How do we set about achieving ‘cultural change’?

Marta started the conversation off by asking what everyone present was already doing at their institutions to engage researchers. Many shared their experiences and some frustrations at pushing good data management habits. One person shared that at his university the initial push toward better data management was achieved by creating and delivering RDM workshops for PhDs and young researchers in the Digital Humanities. These students were already interested in digital preservation, so they were a keen audience. Targeting PhD students and early career researchers may be a more effective strategy because they could develop good data management habits early in their careers. The earlier the intervention, the easier it would (hopefully) be.

Overall, most agreed that directly speaking to researchers is more effective than having initiatives relayed from the top-down. Attendees perceived compliance as a driver rather than a useful stick to persuade researchers to take data management seriously. Even if only a few researchers turned up to data management events, it was still increasing exposure.

Some argued for a multi-prong strategy. Initiatives like the Data Stewards at Delft TU and the Data Champions at the University of Cambridge were perceived as good ways to reach out to researchers in their departments and provide more customized advice. At the same time, having expectations of good data management relayed from on high could help creating greater impetus.

What do we mean by ‘cultural change’?

Naturally, the conversation progressed to what the phrase ‘cultural change’ actually means. It was difficult to determine in 45 minutes what kind of ‘cultural change’ we wanted to see within our different institutions. We started by asking some questions. What were our goals? What would need to happen before we said yes, the culture is changing? Which really meant what do we measure to find evidence of cultural change? Is it better metadata, more awareness of copyright, researchers reaching out to us for help, or an increase in number of grants awarded that would signal an actual change? It would seem that there could be many definitions of ‘cultural change’, but the crucial takeaway is that it is essential to define what your parameters of cultural change will be in the planning stages of any RDM programme.

Where is the evidence?

The conversation progressed to how do we find and gather evidence. With all of the work being done by researchers, librarians, and administrators, how do we know what is actually effective? We cannot state that engaging with researchers (which can be time-consuming) is working without having actual evidence to confirm it. A number of different ideas were discussed, with the time when feedback was gathered being a particular point of variance.

Quantifiable information such as number of datasets deposited, number of datasets downloaded and re-used, and number of grants with a Data Management Plan could be collected. For example, the University of Illinois conducted a detailed analysis of 1,260 data management plans using a controlled vocabulary list and looked at possible correlations between solutions for data management listed in funded and unfunded proposals.

Another method of benchmarking included asking researchers to periodically complete short surveys on data management practice in order to measure any noticeable changes. In that way, an institution can assess whether their engagement strategies work and whether it achieves the desirable effects (improvement of data management practice). Delft, EPFL, Cambridge and Illinois collaborated on development of an agreed set of survey questions. Conducting this same survey across different institutions enables benchmarking and comparison of the different techniques and how effective they are in achieving cultural change in data management. In addition to this survey, the team also interviews some researchers in order to gather additional qualitative data and more detailed insights into data management practice. The hope is that carrying out these quantitative surveys and qualitative interviews periodically will correct for the potential problem of self-selecting participants.

In the future

Ultimately, it turned out that most of those attending the focus group discussion were already working actively to develop systems to measure impact and gather feedback. However, the possibility of carrying out long-term cross-institutional research that would allow comparisons between different data management programmes is very tantalising. The final takeaway from this focus group discussion was that the majority of those attending would be very keen to take part in such research, so watch this space!

Published 18 December 2017
Written by Katie Hughes and Lucy Welch
Creative Commons License

Engaging Researchers with Good Data Management: Perspectives from Engaged Individuals

We need to recognise good practice, engage researchers early in their career with research data management and use peers to talk to those who are not ‘onboard’. These were the messages five attendees at the Engaging Researchers in Good Data Management conference held on the 15th of November.

The Data Champions and Research Support Ambassadors programmes are designed to increase confidence in providing support to researchers in issues around data management and all of scholarly communications respectively. Thanks to the generous support of the Arcadia Foundation, five places were made available to attend this event. In this blog post the three Data Champions and two Research Support Ambassadors who were awarded the places give us the low-down on what they got out of the conference and how they might put what they heard into practise.

Recordings of the talks from the event can be found on the Cambridge University Library YouTube channel.

Financial recognition is the key

Dr Laurent Gatto, Senior Research Associate, Department of Biochemistry, University of Cambridge and Data Champion

As a researcher who cherishes good and reproducible data analysis, I naturally view good data management as essential. I have been involved in research data management activities for a long time, acting as a local data champion and participating in open research and open data events. I was interested in participating in this conference because it gathered data champions, stewards and alike from various British and European institutions (Cambridge, Lancaster, Delft), and I was curious to see what approaches were implemented and issues were addressed across institutions. Another aspect of data championship/stewardship I am interested in is the recognition these efforts offer (this post touches on this a bit).

Focusing on the presentations from Lancaster, Cambridge and Delft, it is clear that direct engagement from active researchers is essential to promote healthy data management. There needs to be an enthusiastic researcher, or somebody that has some experience in research, to engage with the research community about open data, reproducibility, transparency, security; a blunt top-down approach lead to limited engagement. This is also important due to the plurality of what researchers across disciplines consider to be data. An informal setting, ideally driven by researchers and, or in collaboration with librarians, focusing on conversations, use-cases, interviews, … (I am just quoting some successful activities cited during the conference) have been the most successful, and have sometime also lead to new collaborations.

Despite the apparent relative success of these various data championing efforts and the support that the data champions get from their local libraries, these activities remain voluntary and come with little academic reward. Being a data champion is certainly an enriching activity for young researchers that value data, but is comes with relatively little credit and without any reward or recognition, suggesting that there is probably room for a professional approach to data stewardship.

With this in mind, I was very interested to hear the approach that is currently in place at TU Delft, where data stewards hold a joint position at the Centre for Research Data and at their respective faculty. This defines research data stewardship as an established and official activity, allows the stewards to pursue a research activity, and, explicitly, links research data to research and researchers.

I am wondering if this would be implemented more broadly to provide financial recognition to data stewards/champions, offer incentives (in particular for early-career researchers) to approach research data management professionally and seriously, make data management a more explicit activity that is part of research itself, and move towards a professionalisation of data management posts.

Inspiration and ideas

Angela Talbot, Research Governance Officer, MRC Biostatistics Unit and Data Champion

Tasked with improving and updating best practice in the MRC Biostatistics Unit, I went along to this workshop not really knowing what to expect but hopeful and eager to learn.

Good data management can meet with resistance as while it’s viewed as an altruistic and noble thing to do many researchers worry that to make their research open and reproducible opens them to criticism and the theft of ideas and future plans. What I wanted to know are ways to overcome this.

And boy did this workshop live up to my expectations! From the insightful opening comments to the though provoking closing remarks I was hooked. All of the audience were engaged in a common purpose, to share their successes and strategies for overcoming the barriers that ensure this becomes best practice.

Three successful schemes were talked through: the data conversations in Lancaster, the Data Champion scheme at the University of Cambridge and the data stewards in TU Delft. All of these successful schemes had one thing in common: they all combine a cross department/ faculty approach with local expertise.

Further excellent examples were provided by the lightning talks and for me, it was certainly helpful to hear of successes in engaging researchers on a departmental level.

The highlight for me were the focus groups – I was involved in Laurent Gatto’s group discussing how to encourage more good data management by highlighting what was in to for researchers who participate but I really wish I could have been in them all as the feedback indicated they had given useful insights and tips.

All in all I came away from the day buzzing with ideas. I spent the next morning jotting down ideas of events and schemes that could work within my own unique department and eager to share what I had learnt. Who knows, maybe next time I’ll be up there sharing my successes!!

We need to speak to the non-converted

Dr Stephen Eglen, Reader in Computational Neuroscience, Department of Applied Mathematics & Theoretical Physics, University of Cambridge and Data Champion

The one-day meeting on Engaging Researchers in Good Data Management served as a good chance to remind all of us about the benefits, but also the responsibilities we have to manage, and share, data. On the positive side, I was impressed to see the diversity of approaches lead by groups around the UK and beyond. It is heartening to see many universities now with teams to help manage and share data.

However, and more critically, I am concerned that meetings like this tend to focus on showcasing good examples to an audience that is already mostly convinced of the benefits of sharing. Although it is important to build the community and make new contacts with like-minded souls, I think we need to spend as much time engaging with the wider academic community.   In particular, it is only when our efforts can be aligned with those of funding agencies and scholarly publishing that we can start to build a system that will give due credit to those who do a good job of managing, and then sharing, their data. I look forward to future meetings where we can have a broader engagement of data managers, researchers, funders and publishers.

I am grateful to the organisers to have given me the opportunity to speak about our code review pilot in Neuroscience. I particularly enjoyed the questions. Perhaps the most intriguing question to report came in the break when Dr Petra ten Hoopen asked me what happens if during code review a mistake is found that invalidates the findings in the paper? To which I answered (a) the code review is supposed to verify that the code can regenerate a particular finding; (b) that this is an interesting question and it would probably depend on the severity of the problem unearthed; (c) we will cross that bridge when we come to it. Dr ten Hoopen noted that this was similar to finding errors in data that were being published alongside papers. These are indeed difficult questions, but I hope in the relatively early days of data and code sharing, we err on the side of rewarding researchers who share.

Teach RDM early and often

Kirsten Elliott, Library Assistant, Sidney Sussex College, University of Cambridge and Research Support Ambassador

Prior to this conference, my experience with Research Data Management (RDM) was limited to some training through the Office of Scholarly Communication and Research Support Ambassadors programme. This however really sparked my interest and so I leapt at the opportunity to learn more about RDM by attending this event. Although at times I felt slightly out of my depth, it was fascinating to be surrounded by such experts on the topic.

The introductory remarks from Nicole Janz were a fascinating overview of the reproducibility crisis, and how this relates to RDM, including strategies for what could be done, for example setting reproducing studies as assignments when teaching statistics. This clarified for me the relationship between RDM and open data, and transparency in research.

There were many examples throughout the day of best practice in promoting good RDM, from the “Data Conversations” held at Lancaster University, international efforts from SPARC Europe and even some from Cambridge itself! Common ground across all of them included the necessity of utilising engaged researchers themselves to spread messages to other researchers, the importance of understanding discipline specific issues with data, and an expansive conception of what counts as “data”.

I am based in a college library and predominantly work supporting undergraduate students, particularly first years. In a way this makes it quite a challenge to present RDM practices as many of the issues are most obviously relevant to those undertaking research. However, I think there’s a strong argument for teaching about RDM from very early in the academic career to ingrain good habits, and I will be thinking about how to incorporate RDM into our information literacy training, and signposting students to existing RDM projects in Cambridge.

Use peers to spread the RDM message

Laura Jeffrey, Information Skills Librarian, Wolfson College, University of Cambridge and Research Support Ambassador

This inspirational conference was organised and presented by people who are passionate about communicating the value of open data and replicability in research processes. It was valuable to hear from a number of speakers (including Rosie Higman from the University of Manchester, Marta Busse-Wicher from the University of Cambridge and Marta Teperek from TU Delft) about the changing role of support staff, away from delivering training to one of coordination. Peers are seen to be far more effective in encouraging deeper engagement, communicating personal rather than prescriptive messages (evidenced by Data Conversations at Lancaster University). A member of the audience commented that where attendance is low for their courses, the institution creates video of researcher-led activities to be delivered at point of need.

I was struck by two key areas of activity that I could act on with immediate effect:

Inclusivity – Beth Montagu Hellen (Bishop Grosseteste) highlighted the pressing need for open data to be made relevant to all disciplines. Cambridge promotes a deliberately broad definition of data for this reason. Yet more could be done to facilitate this; I’ll be following @OpenHumSocSci to monitor developments. We’re fortunate to have a Data Science Group at Wolfson promoting examples of best practice. However, I’m keen to meet with them to discuss how their activities and the language they use could be made more attractive to all disciplines.

Communication – Significant evidence was presented by Nicole Janz, Stephen Eglen and others, that persuading researchers of the benefits of open data leads to higher levels of engagement than compulsion on the grounds of funder requirements. This will have a direct impact on the tone and content of our support. A complimentary approach was proposed: targeted campaigns to coincide with international events in conjunction with frequent, small-scale messages. We’ll be tapping into Love Data Week in 2018 with more regular exposure in email communication and @WolfsonLibrary.

As result of attending this conference, I’ll be blogging about open data on the Wolfson Information Skills blog and providing pointers to resources on our college LibGuide. I’ll also be working closely with colleagues across the college to timetable face-to-face training sessions.

Published 15 December 2017
Written by Dr Laurent Gatto, Angela Talbot, Dr Stephen Eglen, Kirsten Elliott and Laura Jeffrey
Creative Commons License

Planning scholarly communication training in the UK

In June 2017 a group of people (see end for attendees) met in London to discuss the issues around scholarly communication training delivery in the UK. Representatives from RLUK, UKSG, SCONUL, UKCoRR, Vitae, Jisc and some universities had a workshop to nut through the problem. Possibly because of the nature of the attendees of the group, the discussion was very library-centric, but this does not preclude the need for training outside the library sector. This blog is a summary of the discussion from that day.

Background

The decision to hold a meeting like this came out of the a library skills workshop run at UKSG recently. In ensuing discussions, it was agreed that it would be a good idea to get stakeholders together for a symposium of some description to try and nut out how we could collaborate and provide training solutions for scholarly communication across the sector. There is plenty of space in this area for multiple offerings but we do want to make sure we are covering the range of areas and the types of delivery modes and levels required. In preparation for the discussion the group created a document listing scholarly communication training on offer currently.

What is scholarly communication?

An informal survey of research libraries in the UK earlier this year showed that while all respondents had some kind of service that supports aspects of scholarly communication, only half actually used the term ‘scholarly communication’ to describe those services.

A discussion around the table concluded that the term scholarly communication encompasses a wide range of definitions. Some libraries take the boundary that it refers to post-publication. Others address the pre-publication aspect and meet the need of Early Career Researchers for advice on publishing. Services can focus on the academic’s profile of themselves and their research, or the research lifecycle. In some cases there is a question about whether research data management is part of the equation.

The failure of library schools to deliver

It is fairly universally acknowledged that it is a challenge to engage with library schools on the issue of scholarly communication, despite repositories being a staple part of research library infrastructure for well over a decade. There are a few exceptions but generally open access or other aspects of scholarly communication are completely absent from the curricula. (Note: any library school that wishes to challenge this statement, or provide information about upcoming plans are welcome to send these through to info@osc.cam.ac.uk)

This raises the question – if library schools are not providing, how do we recruit and train the staff we need? Indeed, who are we actually recruiting? Is it essential for staff to have a library degree, or experience in an academic library? Or are our requirements more functional such as the ability to manipulate large data sets, or experience working with academics, or an understanding of the Higher Education environment?

While libraries are starting to employ post-graduate researchers because they can lend skills to the library, library culture is a consideration. Employing researchers who are not librarians has the benefit of bringing in expertise from outside, but there are challenges to integrate their work into the library culture. We need to look at competencies in terms of the structure and size of the organisation, both for current staff and staff of the future.

In the absence of scholarly communication instruction within the basic qualification, skills training in this space would appear to need to be addressed at the profession level.

One possible route to prepare the next generation is offering some modular approach of on the job learning with very practical experience. An option could be to work with people who have come from outside the library space. Given libraries seem to be starting to bring skill sets in, we need to consider how this sits with the existing profession.

Audiences and their training needs

The goal of the meeting was to resolve what kinds of training the sector needs, for whom and how it is delivered. For example, with many general library staff there is a basic need to understand the issues with scholarly communication. The number one question is ‘what is scholarly communication’? The possibly it is enough for these people to just be familiar with the terminology.

It is possible we need lots of short courses on the general topic of: this is what OA is, basics of RDM etc (that could potentially be delivered online), but probably fewer more complex courses on issues like analysing publisher and funder policies. There are also debates and higher order areas which require face to face debate.

  • Front facing staff
    • Need an overview so the language is familiar and they can refer queries on
  • People working in scholarly communication
    • Day to day practicalities of funder open access compliance
  • Specialist roles in scholarly communication
    • Specific areas
  • Senior managers
    • Very much need a refresher so they can help their staff.
    • Similar overview training, leadership is around the advocacy
    • Need conceptual framework for scholarly communication – how do the technical parts sit together for the infrastructure and governance of institutions
    • Stakeholder management skills.

Skill sets in scholarly communication

It was agreed that budgetary, presentation and negotiation skills are needed in this area as general skills. When it comes to specialist skills these include:

  • Research Integrity
  • Bibliometrics
    • Involved in providing specialist advice on metrics within a school discussion
    • Providing advice on impact
  • Pushing the open research agenda
  • Academic reward structure
  • Technical and infrastructure eg: integrating ORCIDS etc

Considerations – Lack of perceived need?

There appears to be a problem with a lack of perceived need for training in this space. We are encountering issues where people in libraries are saying ‘I don’t think this is our job’. This points to what should we be presenting librarianship as – what kind of people do we want in the profession? A ‘traditional librarian’ of 20 years ago is not the same job now, the skills are different. Today much of an academic librarian’s job is about winning over people who don’t want to hear the message. It is possible there does need to be a different sort of person who is pushing an open access agenda.

There have been other innovations in library work that required engaging different behaviours and tasks in the past. For example, is this move towards a scholarly communication future different from when the discovery search was introduced? The eResources experience is similar in terms of new competencies required in the profession. However the difference in the scholarly communication environment is there is an external driver – we need to understand the politics of how open access can move forward in the UK.

Considerations – budgets

There is a mismatch between what people would love to have, what can be designed and what people can afford. Anecdotally the group heard that training budgets are really squeezed so priority and focus might be heavily influenced by this, with geography and travelling costs being central to decisions.

The group discussed the need to make training accessible to all. Even free events can be prohibitive in terms of travel, and hosting them in off-peak periods can be helpful with costs. The blockage is not just money, it includes time – in terms of loss of a team member while they are away. This is particularly problematic if scholarly communication is only a part of their job. Most of the need comes from really small institutions where the work is part of a bigger role, however that is where there is little money. This also raises challenges for the time available for those people to self educate.

UKSG run events in London which is expensive for organisations north of London to attend. To increase participation UKSG are now trying to put regional events on, and have shifted their training to a webinar programme rather than face to face.

SCONUL has done basic copyright training and this has thrown up price sensitivity. One solution is trying to keep it local, and members can volunteer staff in kind.

One option could be online training where participants log on at a certain time once a week for 10 weeks. Many of the people in scholarly communication work in universities, and have distance education software available to them. An alternative is having courses done in house – that could part of a modular package (but how do you link this?). The course content needs to be agnostic enough to be useful (not discussing DSpace or PURE for example) before delving into institutional specifics. Make it modular with core principles and then have options.

There was a suggestion that we create a nonprofit making shared collaborative service. The costs to developing this type of deliverable include the development of the training materials, infrastructure costs, room hire, catering etc. Can we make it all online and available? This could work if it were modular.

Next steps

We have not yet bottomed out the need yet – perception of needs at the practitioner level and senior management might be different. Cost is an issue here. Universities need to work out how much it costs to do in-house training – what is the opportunity cost to employ a staff member without experience or training and then get them up to speed?

It would be useful to have an understanding of what training is happening within institutions. What subjects/topics are being taught, who is doing it, what language is being used, is there a dedicated staff member. Where else do people get information and support?

The general plan is to reconvene in September.

Useful Resources

Skill sets analyses

Here are links to work that has already been done on the required skill sets:

Organisations providing or coordinating training

Organisations are running similar events and then participants have to choose what to focus on. If we divvy it up across the sector it might help the situation.

The Society for College, National and University Libraries (SCONUL) does basic copyright training. There is more focus on the leadership end of the equation. The Collaboration Strategy Group is considering a shared service. People come from non traditional groups and this reflects a broader skills sets required in libraries than traditional library courses give you. SCONUL are about to scope out where those services might be and try to identify needs into the future. There are challenges are in recruiting people given the slightly moralistic nature of library culture and whether they are welcoming of people from different background. How do we promote, retain and incentivise people who may not come from this area?

Research Libraries UK (RLUK) don’t do direct training, but they do have programmes of works and networks around these issues. The RLUK board recently had a meeting to look at a new strategy – updating the existing 2014-2017 RLUK Strategy. They are looking at the bigger picture for scholarly communication – the infrastructure challenges, the bigger picture related to licensing and costs and how to leverage members in the consortia. Their role is very much supporting and helping out.

UK Serials Group (UKSG) runs a conference programme. One day events are a mix of standing repeated courses and one off sessions. In conferences often the breakout sessions are the things that people find really valuable. These include soft skills like mindfulness in leadership. The audience tends to be practitioners, people in their mid-career. Traditional areas such as library have been focused around collection management because that is where publishers are. But it is not just about traditional publishing. They are our members and that is moving our agenda to meet those needs. UKSG cannot get anywhere in contributing to university publishing courses. Libraries are starting to employ people who have publishing backgrounds.

The Association of Research Managers and Administrators (ARMA) has special interest groups in open access. (Note: ARMA were invited to this meeting but unfortunately couldn’t attend.)

The Chartered Institute for Library and Information Professionals (CILIP) conducts training at a local level. It was agreed we can’t have the conversation without having CILIP in the room – they are wanting to offer more support for academic libraries and seem to be recognising that the library schools program for CILIP is not the be-all and end-all any more. This is partly why they have developed a recognised trainer programme. (Note: CILIP were invited to this meeting but unfortunately couldn’t attend.)

Representatives attending the discussion

  • Helen Dobson – Manchester University
  • Danny Kingsley – Cambridge University
  • Claire Sewell – Cambridge University
  • Anna Grigson representing UKSG
  • Fiona Bradley – RLUK
  • Ann Rossiter – SCONUL
  • Katie Wheat – Vitae
  • Sarah Bull – UKSG
  • Stephanie Meece -UKCoRR
  • Frank Manista – Jisc
  • Helen Blanchett – Jisc (a member of the group coordinating the meeting, but was unable to attend on the day)

ARMA and CILIP were also invited but were not able to send a representative.

Published 15 August 2017
Written by Dr Danny Kingsley