FRSNZ: a FAIR discussion?
In this section for submissions by our Fellows, Professor Rupert Sutherland shares his own experience as a researcher in Aotearoa to encourage the fellowship to develop a critical lens towards the nation's system for sharing data.
Every FRSNZ aspires to conduct excellent research and most hope that their work produces value for New Zealand (and all of humanity). I do. In this article I tell a short true story that led me to believe our science system in New Zealand could do a lot better; and we could do a lot better. I hope I can persuade you to join with me on a journey to improve research integrity and increase public value; and to improve the quality of communication within our fellowship.
In late 2017 I spent nine weeks in the Tasman Sea on a research vessel collecting sediment cores from deep beneath the seabed. The sediment preserves a wonderful record of how our continent of Zealandia formed and how its climate and biota evolved [1]. Our analysis required comparisons to archival seabed data held in databases of three institutions. It took only a few hours to download data from institutions in USA and Australia, but I still don’t have agreement from the New Zealand institution to even see their data. I patiently progressed through the process of asking at many levels until I eventually got a definitive ‘no’ from executive management. No reason was given, and I can’t think of any reasonable one. Even if they did let me see ‘their’ data, prior experience suggests the data use agreement would be extremely restrictive. The public investment in those data is measured in 100s of millions of dollars.
There is a twist to this story: a paper describing the data I was interested in was published just a few months before I asked for them. I went to the journal supplementary material, but found no data were lodged. I checked journal guidelines and they did not require authors to provide data. “New Zealand Journal of Geology and Geophysics [is] a publication of the Royal Society Te Apārangi” appears on the web interface to the journal. None of our (RSNZ) journals require authors to make data available to peers for scrutiny. Since I raised this issue last year, RSNZ journals increased their requirement to “expects data sharing”, but that is not a high bar. It is still possible to say no, or create barriers or delays, and to restrict or stop reuse.
There are two reasons why I think none of this is OK: integrity and value.
Reproducibility and replicability are cornerstones of integrity in modern science [2]. The web of interdependencies in research means it is essential to find error or weakness at the earliest possible stage, so that future investment is not wasted. It is difficult to find error or weakness because research is highly technical and few people have the time or competence. That is why the broader community has to be given a chance. Quality assurance is underpinned by integrity of data, completeness of metadata, and full documentation of methods. We do not require authors in RSNZ journals to provide data, metadata, and code to the broader community, and we have no code of ethics that demands members conform to international best-practice, which is defined by the stated positions of societies similar to us, e.g. [3]. We should not accept this deficiency in integrity.
Access to data, metadata, and code has always been necessary for quality assurance, but in the last decade it has become increasingly evident that much greater scientific and public value may be obtained by reanalysis of machine-readable data. Computer codes that mine many sources are now capable of looking for hidden relationships that were previously impossible to find. Data need to be findable, accessible, interoperable (metadata meet some common standard), and reusable (license terms allow reanalysis and reuse) = FAIR [4]. Some practical barriers [2, 5] and negative incentives remain [6], but research shows that scientists personally benefit from participating in open science [7]. The recognition of enhanced public outcomes has led to a global move towards FAIR data and hence repositories for open data and code now exist in many countries [2, 4, 5, 8, 9]. The large public value proposition [8, 10] makes it certain (to me) that open access to most research will become reality during the next decade. We are an anomaly among developed nations.
I understand why we are in this situation in New Zealand, but I don’t accept that it is OK. The principles of integrity and value in science are important, but sometimes conflict with profit incentives of individuals or institutions. How did we come to accept such a loss of integrity and public value in our public institutions? Surely, as a coordinated fellowship we should be the watchdogs of this?
The fellowship of the Society is not currently a watchdog or driver of change. We barely ever communicate with each other, which is a shame because that is how and why the Royal Society in England initially formed. I applaud the Society for starting this newsletter. It is necessary, but I don’t think it is sufficient. If we identify an issue like this, some of us will have to work together, to pool knowledge and experiences, and decide what the best way to proceed is. I suggest that we also need an online workspace. Now is the perfect time to start – we have all been forced into online workspaces over the last few months and we know they can work. This newsletter will be a great way to communicate summaries of key developments or develop awareness of new threads, but we need a space where we can initiate and moderate our own discussions within our fellowship. If you agree, then please provide feedback to Royal Society Te Apārangi and let’s make it happen.
About the author
Professor Rupert Sutherland is a Professor in Geophysics and Tectonics at the School of Geography, Environmental and Earth Science at Victoria University of Wellington.
He can be contacted by email, or by the following address:
Victoria University of Wellington, PO Box 600, Wellington, New Zealand
If Fellows would like to respond or comment on the topics in the Fellows' Forum newsletter discussion section, please email the academy and your letter will be included in the next newsletter.
References
1. Sutherland, R. and G. Dickens, Expedition reveals the violent birth of Earth’s hidden continent Zealandia, forged in a ring of fire. The Conversation, 2020. https://theconversation.com/expedition-reveals-the-violent-birth-of-earths-hidden-continent-zealandia-forged-in-a-ring-of-fire-130860.
2. US National Academies of Sciences, E., Medicine, Reproducibility and replicability in science. 2019: National Academies Press.
3. US National Academies of Sciences, E., Medicine, Fostering Integrity in Research. 2017, Washington, US: US National Academies of Sciences, Engineering, Medicine. 327.
4. Wilkinson, M.D., et al., The FAIR Guiding Principles for scientific data management and stewardship. Scientific data, 2016. 3.
5. The Royal Society, Science as an open enterprise. The Royal Society Science Policy Centre report 02/12. 2012, London, UK: The Royal Society. 105.
6. Nosek, B.A., et al., Promoting an open research culture. Science, 2015. 348(6242): p. 1422-1425.
7. McKiernan, E.C., et al., Point of view: How open science helps researchers succeed. Elife, 2016. 5: p. e16800.
8. G20, G20 Leaders’ Communique Hangzhou Summit, STATEMENT/16/2967. 2016, Hangzhou, China: G20.
9. OECD, Promoting access to public research data for scientific, economic, and social development. 2003, Paris, France: Organisation for Economic Co-operation and Development. 38.
10. OECD, Policy initiatives to enhance the impact of public research: promoting excellence, transfer, and co-creation. OECD Science Technology and Industry Policy Papers, 2019. 81: p. 1-26.