Preparing for the end of COAF

The Open Access team are getting ready for the end of Charity Open Access Fund (COAF), which is due to dissolve on 30th September 2020.  

From 1st October 2020 onward, there are going to be changes to the block grants that we receive, and as a result, there will be a change in our policies on whether or not we can cover researchers’ article processing charges (APCs).  

We have outlined how researchers should go about securing funding for the APC’s below: 

Funder name Are article processing charges covered by a block grant? How do I pay for my article processing charge? 
UKRI Yes No change: researchers should continue to upload their paper to us for a funding decision
Wellcome Trust Yes No change: researchers should continue to upload their paper to us for a funding decision
Cancer Research UK Yes No change: researchers should continue to upload their paper to us for a funding decision
British Heart Foundation No- institutions apply to BHF for reimbursement No change: researchers should continue to upload their paper to us for a funding decision
Blood Cancer UK No- authors must include cost in their grant application  1. For payment, contact research@bloodcancer.org.uk
2. Upload your paper to ensure REF compliance. 
Parkinson’s UK No- authors must include cost in their grant application  1. For payment, contact researchapplications@parkinsons.org.uk,
2. Upload your paper to ensure REF compliance. 
Versus Arthritis No – authors must request support direct from funder  1. Use funder’s Grant Tracker for OA support,
2. Upload your paper to ensure REF compliance. 
Multiple funders acknowledged  If your paper includes funding from UKRI, Wellcome Trust, Cancer Research UK or British Heart Foundation then we may be able to help with the APC. Researchers should upload their paper to us for a funding decision

There is no change in the funder’s open access policies for the rest of 2020. However, there are significant changes due in 2021, specifically to Wellcome Trust and Cancer Research UK.  

We have outlined the policy changes in the table below: 

Funder name Change? Outline of policy 
Wellcome Trust Changesee new policy document   1. Policy covers original research articles, 
2. Policy applies to papers submitted for publication after 1/1/2021, 
3. Papers must be made immediately open access (no embargo allowed) in Europe PMC, 
4. Papers must be published with a CC BY licence, 
5. Papers must be published in a journal that is indexed in DOAJ (Wellcome will no longer cover APCs for subscription journals)
6. The authors must retain their copyright. 
Cancer Research UK Changesee new policy document 1. Policy covers original research articles, 
2. Policy applies to all papers after 1/1/2021, 
3. Papers must be made immediately open access (no embargo allowed) in Europe PMC,
4. Papers must be published with a CC BY licence. 
Multiple funders acknowledged  Any papers acknowledging Wellcome Trust or Cancer Research UK must be compliant in order to access funds. 

To summarise:

From 1 October 2020, authors should continue to submit their papers to the Open Access Team as usual via our website. The Open Access Team will continue to advise on the best course of action to meet funder requirements, but we may not always be able to pay APCs. 

The funders’ policies remain the same until 1st January 2021. We advise authors covered by Wellcome Trust and Cancer Research UK to familiarise themselves with the changes to their funder’s open access policies, which are outlined in COAF’s table

Data sharing and reuse case study: the Mammographic Image Society database

The Mammographic Image Society (MIAS) database is a set of mammograms put together in 1992 by a consortium of UK academic institutions and archived on 8mm DAT tape, copies of which were made openly available and posted to applicants for a small administration fee. The mammograms themselves were curated from the UK National Breast Screening Programme, a major screening program that was established in the late 80s offering routine screening every three years to women aged between 50-64.

The motivations for creating the database were to make a practical contribution to computer vision research – which sought to improve the ability of computers to interpret images – and to encourage the creation of more extensive datasets. In the peer-reviewed paper bundled with the dataset, the researchers note that “a common database is a positive step towards achieving consistency in performance comparison and testing of algorithms”.

Due to increased demand, the MIAS database was made available online via third parties, albeit in a lower resolution than the original. Despite no longer working in this area of research, the lead author, John Suckling – now Director of Research in the Department of Psychiatry, part of Cambridge Neuroscience –  started receiving emails asking for access to the images at the original resolution. This led him to dig out the original 8mm DAT tapes with the intention of making the images available openly in a higher resolution. The tapes were sent to the University Information Service (UIS), who were able to access the original 8mm tape and download higher resolution versions of the images. The images were subsequently deposited in Apollo and made available under a CC BY license, meaning researchers are permitted to reuse them for further research as long as appropriate credit is given. This is the most commonly used license for open datasets and is recommended by the majority of research funding agencies.

Motivations for sharing the MIAS database openly

The MIAS database was created with open access in mind from the outset. When asked whether he had any reservations about sharing the database openly, the lead author John Suckling noted:

There are two broad categories of data sharing; data acquired for an original purpose that is later shared for secondary use; data acquired primarily for sharing. This dataset is an example of the latter. Sharing data for secondary use is potentially more problematic especially in consortia where there are a number of continuing interests in using the data locally. However, most datasets are (or should be) superseded, and then value can only be extracted if they are combined to create something greater than the sum of the parts. Here, careful drafting of acknowledgement text can be helpful in ensuring proper credit is given to all contributors.”

This distinction – between data acquired for an original purpose that is later shared for secondary use and data acquired primarily for sharing – is one that is important and often overlooked. The true value of some data can only be fully realised if openly shared. In such cases, as Suckling notes, sufficient documentation can help ensure the original researchers are given credit where it is due, as well as ensuring it can be reused effectively. This is also made possible by depositing the data on an institutional repository such as Apollo, where it will be given a DOI and its reuse will be easier to track.

Impact of the MIAS database

As of August 2020, the MIAS database has received over 5500 downloads across 27 different countries, including some developing countries where breast cancer survival rates are lower. Google Scholar currently reports over 1500 citations for the accompanying article as well as 23 citations for the dataset itself. A review of a sample of the 1500 citations revealed that many were examples of the data being reused rather than simply citations of the article. Additionally, a systematic review published in 2018 cited the MIAS database as one of the most widely used for applying breast cancer classification methods in computer aided diagnosis using machine learning, and a benchmarking review of databases used in mammogram research identified it as the most easily accessible mammographic image database. The reasons cited for this included the quality of the images, the wide coverage of types of abnormalities, and the supporting data which provides the specific locations of the abnormalities in each image.

The high impact of the MIAS database is something Suckling credits to the open, unrestricted access to the database, which has been the case since it was first created. When asked whether he has benefited from this personally, Suckling stated “Direct benefits have only been the citations of the primary article (on which I am first author). However, considerable efforts were made by a large number of early-career researchers using complex technologies and digital infrastructure that was in its infancy, and it is extremely gratifying to know that this work has had such an impact for such a large number of scientists.”. Given that the database continues to be widely cited and has been downloaded from Apollo 1358 times since January 2020, it is still clearly the case that the MIAS database is having a wide impact.

The MIAS Database Reused

As mentioned above, the MIAS database has been widely reused by researchers working in the field of medical image analysis. While originally intended for use in computer vision research, one of the main ways in which the dataset has been used is in the area of computer aided diagnosis (CAD), for which researchers have used the mammographic images to experiment with and train deep learning algorithms. CAD aims to augment manual inspection of medical images by medical professionals in order to increase the probability of making an accurate diagnosis.

A 2019 review of recent developments in medical image analysis identified lack of good quality data as one of the main barriers researchers in this area face. Not only is good quality data a necessity but it must also be well documented as this review also identified inappropriately annotated datasets as a core challenge in CAD. The MIAS database is accompanied by a peer-reviewed paper explaining its creation and content as well as a read me PDF which explains the file naming convention used for the images as well as the annotations used to indicate the presence of any abnormalities and classify them based on their severity. The presence of this extensive documentation combined with it having been openly available from the outset could explain why the database continues to be so widely used.

Reuse example: Applying Deep Learning for the Detection of Abnormalities in Mammograms

This research, published in 2019 in Information Science and Applications, looked at improving some of the current methods used in CAD and attempted to address some inherent shortcomings and increase the competency level of deep learning models when it comes the minimisation of false positives when applying CAD to mammographic imaging. The researchers used the MIAS database alongside another larger dataset in order to evaluate the performance of two existing convolutional neural networks (CNN), which are deep learning models used specifically for classifying images. Using these datasets, they were able to demonstrate that versions of two prominent CNNs were able to detect and classify the severity of abnormalities on the mammographic images with a high degree of accuracy.

While the researchers were able to make good use of the MIAS database to carry out their experiments, due to the inclusion of appropriate documentation and labelling, they do note that since it is a relatively small dataset it is not possible to rule out “overfitting”, where a deep learning model is highly accurate on the data used to train the model, but may not generalise well to other datasets. This highlights the importance of making such data openly available as it is only possible to improve the accuracy of CAD if sufficient data is available for researchers to carry out further experiments and improve the accuracy of their models. ­

Reuse example: Computer aided diagnosis system for automatic two stages classification of breast mass in digital mammogram images

This research, published in 2019 in Biomedical Engineering: Applications, Basis and Communications, used the MIAS database along with the Breast Cancer Digital Repository to test a CAD system based on a probabilistic neural network – a machine learning model that predicts the probability distribution of a given outcome –  developed to automate classification of breast masses on mammographic images. Unlike previously developed models, their model was able to segment and then carry out a two-stage classification of breast masses. This meant that rather than classifying masses into either benign or malignant, they were able to develop a system which carried out a more fine-grained classification consisting of seven different categories. Combining the two different databases allowed for an increased confidence level in the results gained from their model, again raising the importance of the open sharing of mammographic image datasets. After testing their model on images from these databases, they were able to demonstrate a significantly higher level of accuracy at detecting abnormalities than had been demonstrated by two similar models used for evaluation. On images from the MIAS Database and Breast Cancer Digital Repository their model was able to detect abnormalities with an accuracy of 99.8% and 97.08%, respectively. This was also accompanied by increased sensitivity (ability to correctly classify true positives) and specificity (ability to correctly classify false negatives).

Conclusion

Many areas of research can only move forward if sufficient data is available and if it is shared openly. This, as we have seen, is particularly true in medical imaging where despite datasets such as the MIAS database being openly available, there is a data deficiency which needs to be addressed in order to improve the accuracy of the models used in computer-aided diagnosis. The MIAS database is a clear example of a dataset that has enabled an important area of research to move forward by enabling researchers to carry out experiments and improve the accuracy of deep learning models developed for computer-aided diagnosis in medical imaging. The sharing and reuse of the MIAS database provides an excellent model for how and why future researchers should make their data openly available.

Published 20th August 2020
Written by Dominic Dixon

CCBY icon

Cambridge response to the UKRI open access policy review

Open access is transforming scholarly communication, and both the University and its Press are fully committed to the transition to open access publishing without embargo. It is inspiring us to think more deeply about how the research publishing ecosystem can be improved to the benefit of all society.

The open access policy review being conducted by UK Research and Innovation (UKRI) will have a major impact on how publicly funded research in the UK is published. The UK already has a strong commitment to open access, and we look forward to the new UKRI policy dramatically speeding up the country’s transition to open access.

Cambridge unites a world-leading research university, with a world-renowned Press and Library. We believe there is strength in this partnership, including the ability to challenge and test solutions that must work for academics, funders, publishers and research institutions. Our joint response to the UKRI policy review reflects the range of perspectives across the University and highlights some of the challenges and opportunities we face as an academic university and publisher.

In brief:

  • There are many aspects of the proposed URKI policy that we support without reservation. For example, authors should retain their copyright, journals and publishers should be more transparent about their services and costs, and key metadata, such as funder and grant information and author IDs, are vital for efficient scholarly communication and research evaluation infrastructures.
  • There is a conflict between the need for sustainable journal publishing models that provide access to the final published article and affordability for research-intensive universities. Collectively we believe that this contradiction in approach is not sustainable and necessitates a UKRI policy that is more flexible in the short term while supporting a much bolder shift in publishing practice that will require significant changes from all stakeholders. The Library and the Press are working together to explore bold innovation and disruption for scholarly communications built round a shared commitment to the goals of open research.
  • There are also areas where we agree that allowances must be made for the different needs of different research communities. While all research communities must be able to benefit from OA, flexibility on details such as Creative Commons licenses and third party content is needed to allow research, and international collaboration, to flourish. There are concerns from academics, Library and the Press, for example, about the potential for requiring open access to all monographs in the REF-after-REF 2021 in the absence of funding for publishing these monographs, around the cost implications of requiring open access to articles and monographs that include third party content and around unintended consequences for early career researchers in certain disciplines.
  • For books, we need the time and freedom to find scalable, sustainable approaches to OA. No model has been found so far that would allow us to publish large numbers of high-quality OA books at the global scale and reach of the Press. The impact of making pre-final versions of books open access after an embargo is inadequately understood, undesirable from the perspective of researchers in particular disciplines and may be economically unrealistic (because we believe book purchasing habits will change significantly under a delayed-OA approach). While new approaches are explored, we suggest a couple of options for UKRI to consider adopting: (i) broadening the definition of ‘open’ to include ‘free to read’ and (ii) allowing books to be published under a ‘transformative programme’, perhaps along the lines of the Subscribe To Open model for journals.
  • For journal articles, we cannot ignore an essential paradox. On the one hand, zero embargo Green OA depends upon subscriptions which are becoming ever more unsustainable as more content becomes OA. On the other hand, many research-intensive organizations are unable to pay the costs of their publishing without subsidies from subscribers around the world. Our academic University would need to comply with the proposed UKRI policy predominantly through the Green OA route, while CUP needs to transition to Gold OA. To resolve this paradox during a world-wide shift to full open access, UKRI must make two transitionary allowances: modest embargoes can be applied by publishers to support the subscriptions that sustain Green OA, and Gold OA in hybrid journals must continue to be supported. We want to see a scholarly communications landscape that has diversity reflecting the breadth scholarship across the disciplines, including smaller publishers and learned societies that require support in the transition to Open Access.

As we said earlier, we look forward to the new UKRI policy dramatically speeding up the UK’s transition to OA. We hope that the fine details of the policy will allow us to fully play our part in the transformation.

This post has been developed jointly by Cambridge University Libraries and Cambridge University Press and has also been shared at https://www.cambridge.org/core/blog/?p=36924.