President's Address 2019 on redefining research excellence

Tēnā koutou katoa

Nau mai haere mai ki tēnei pō hei whakanui i ngā Takahoa a Te Apārangi.
He tino pai te kite i a koutou katoa.

National academies like Royal Society Te Apārangi have been around for over 350 years. They are based on the premise that those recognised as intellectual leaders by an Academy have a duty for collective endeavour that serves the people of their nation in a selfless manner. We take that premise seriously at Te Apārangi and it guides the wide range of programmes we run. Those of us who have the privilege of serving as President of such academies have a thought leadership role, and it is in that capacity that I am addressing you this evening.

I’m going to speak to the topic of Excellent Research. Now you might think that this is a relatively easy discussion to have; that in my day job as Provost at Victoria University of Wellington Te Herenga Waka I should encourage my academic colleagues to aspire to greater heights, publishing in the high impact journals in their fields, supervising successful doctoral students, and generating external research income. In my role as President of Royal Society Te Apārangi, we should recognise outstanding researchers with our awards and medals, acknowledging the very best with the elite status of Fellowship thereby ‘rewarding excellence in the broad areas of science, technology and humanities’ to use the language from the Act that established the Royal Society.

What I want to show you tonight is that research excellence comes in multiple forms, particularly in an era of impact and advancement. Many of you here tonight will rely on research outputs to support your wider activities, and it is this shift towards engagement and relevance that is challenging established conceptions of research excellence in what is often known as a ‘grand challenge’ world.

Let me begin by putting my social science hat on and contextualising the claim that research excellence comes in multiple forms in a discussion about changing modes of academic governance. In recent years these debates have been dominated by accounts of the so-called ‘neoliberal university’ or perhaps more specifically (to use Slaughter and Rhoades term) ‘academic capitalism’. Attention has been consistently drawn to processes of globalisation (for example, the rise of international university rankings such as THE, QS and APRU), the ever-increasing encroachment of ‘audit culture’ (for example, the proliferation of research assessment exercises such as PBRF, REF and ERA), and the rise of highly individualistic academic subjectivities, including the so-called ‘celebrity academic’ particularly in places like the United States.

Political economists argue that these processes are underpinned by the changing political economy of knowledge production, including increasing competition as more institutions compete for ever declining public resources, the growth of a new set of actors who are moving into research and teaching territories historically occupied by universities, the over production of PhD students, and a tightening labour market resulting in a precariously employed cohort of early career researchers. The result, it is argued, is an increasingly heterogenous research sector marked by new hierarchies and exclusions within and between institutions.
It is easy to argue (and many do) that in this context research excellence has become narrowly defined. Excellence means publishing in high profile international journals and prestigious (largely US-based) university presses. For non-Anglo colleagues it also means publishing in English. Research excellence is increasingly quantified through PBRF grades, H factors, citation scores, journal impact factors and the like. Many researchers feel they have been incentivised to perform this version of research excellence through labour market dynamics and promotions processes. They argue that to succeed in these terms means moving up the research hierarchy by strategically publishing and focusing narrowly on the research component of their roles, to the detriment of teaching in our universities and wider forms of organisational citizenship essential to making our research ecosytem work.

New Zealand researchers have a particular beef in this regard; they feel that those who focus on local topics and issues are undervalued, whereas those who are more internationalist in their aspirations get more recognition. I’ve lost count of the number of times I’ve been told that colleagues who research New Zealand topics can’t get published internationally.

My view is that this analysis is extremely unhelpful. If we believe this description of our research sector, then my job is to foster ever greater competition and heightened individualism at exactly the time we need researchers able to offer multi-disciplinary and cross-sectoral leadership in the context of a so-called ‘grand challenge’ world. I am particularly worried about the model of PhD training we foster if we continue to act on this analysis. To caricature; to produce an excellent researcher we subject people to a training process that plays on all their individual insecurities, including putting them in a room by themselves for three or four years to write their thesis. At the end of that process we expect them to be good collaborative subjects, and able to effectively participate in the cohorts and communities needed in today’s research world.

We need to ask different questions about the nature of research excellence. My view is that the highly individualised and competitive features of the research landscape that I have set out above are both partial and dated. Research colleagues in CRIs, independent research organisations, the private sector and iwi organisations will already be attuned to a much more porous and heterogenous research landscape than that described above. We might draw attention to the growth of new research relationships with industry, government and cultural institutions. Women’s studies, indigenous studies and development studies have also long been sites from which calls for new relationships with external partners have emerged and been enacted. We should be particularly aware of this in Aotearoa New Zealand given the importance and influence of kaupapa Māori in shaping knowledge formations and research practices in this country.

In my view, insufficient attention has been paid to the nature of the contemporary research landscape and the way in which it is challenging traditional conceptions of research excellence. These new forms of research require the building of new connections between researchers and external ‘stakeholders’. They are often premised on deep inter-personal relations which aim to produce socially significant knowledge through a reciprocal understanding of the expertise and insights of both partners in the research process. In the UK this terrain is known as co-produced research, here in New Zealand it is more often described as co-designed research. It is also the aspiration that underpins ambitions for Mātauranga Māori.

The impetus for these new research relationships is driven by the rise of what sometimes gets called ‘grand challenges’. Taking the example of climate change, we know even the climate scientists argue the major research challenges are now regulatory, social and behavioural, requiring new relationships and new modes of working. This is driving both a new emphasis on interdisciplinarity and a new imperative for cross-sectoral working. One result is that co-design is increasingly commonplace in funding calls and research initiatives like our National Science Challenges, marking an aspiration for collectively produced knowledge. New funding streams are opening up, designed to promote collaborative endeavours and co-production between researchers and external stakeholders.

These moves across disciplines and organisations raise new questions that concern researchers and practitioners alike, including the problematising of established research approaches. Whereas traditional research practice encourages clearly defined research identities and professional relationships, co-designed research requires reflexive engagements and emotional labour. It raises important ethical questions and generates new forms of accountability. The focus of research management shifts from defining functional relationships to managing interpersonal dynamics. It also marks a move away from the presumed masculinity of traditional leadership styles valued in the academy. Our research organisations themselves are being reconfigured through the changing infrastructures and imperatives of co-designed research. Finally, this new terrain raises new questions about what constitutes ‘excellent’ research.

Let me give you three examples to help make this tangible.

Let me begin with my new role; namely that of President of Te Apārangi. This august organisation is now 151 years old. We took the opportunity of our 150th anniversary to think hard about how we remain relevant for the next 150 years. Part of this discussion was about becoming a more inclusive organisation; better engaging with the full range of disciplines that makes up Te Apārangi and recognising the diversity of excellent colleagues other than the male scientists who have historically tended to dominate the ranks of our Fellowships and Medals and Awards winners. This has meant acknowledging the research excellence comes in multiple forms, and considering what this might mean for our nominations processes. For example, we now understand that to recognise the excellence of women and Māori it is important to take a more holistic approach, recognising that mentoring, stakeholder engagement, and impact beyond narrow disciplinary research fields may be as important as the traditional indicators such as numbers of research publications.

It has also encouraged us to look hard at how we have interpreted our Act. The Royal Society of New Zealand Act allows us to award Fellowship for “distinction in research or the advancement of science, technology, or the humanities”. While this allows for Fellowship on two grounds – research or advancement - historically we have only accepted nominations for distinction in research. We are now moving the same way as many international academies and actively exploring the second route of advancement. Examples here could range from major creative works through to significant inventions. If Janet Frame was alive would we award her Fellowship alongside her recognition by the American Academy of Arts and Letters? If John Britten was alive would we recognise him for the invention of the Britten motorcycle? We are unequivocal that both the research and advancement routes excellence must be demonstrated through intellectual endeavour, but to give effect to that we need to allow for different forms of evidence. So what we see here is a more multi-faceted conception of excellence; one that acknowledges that research excellence takes multiple forms.

I have focused on discussions of Fellowship, but hopefully you will see that this starting point - research excellence takes multiple forms - has implications for many other Royal Society activities. How can we work more effectively with Māori research organisations? What does this means for our early career researchers and the ways in which we engage with them? How should we structure our expert advice panels? How can we ensure that equity and diversity are an integral part of all that we do? Whatever the answers to these questions, there can be no doubt that Te Apārangi will look very different in the next 150 years than it has in the past.

Let me give you another example. I am also on a Main Panel for the UK REF 2021 exercise – the UK equivalent of PBRF. The Main Panels set the criteria for this exercise and here again we find a reconfigured discussion about research excellence. The REF methodology has three parts; outputs, impact and environment. This is the second REF to explicitly consider the impact of the research being assessed (with an increased percentage resting on success in this area) and the emphasis on environment is increasingly focused on the work units do to ensure success for all, including equity and diversity, mentoring and the like. Let’s look at this in a little more detail as there are some important learnings herein.

The definition of excellence for the outputs has largely remained the same, however it is important to note that this definition is not metricised. Peer review remains the means by which originality, significance and rigour of outputs is assessed.

Perhaps more interesting for our purposes is the question of how the excellence of ‘impact’ is assessed in REF. The key terms here are reach and significance. For those of you unfamiliar with this assessment exercise, impact is assessed through narrative case studies and underpinning evidence. Impact case study writers, journalists, film-makers, bloggers, public engagement experts have all had roles to play in ensuring the research was captured, had impacts, and that these impacts could be evidenced as ‘excellent’. And just in case you think impact is an instrumentalist approach to research, I’d encourage you to explore the highest scoring impact case studies from last time. On my own geography panel these included the Living Wage and a project working with the victims of sex trafficking, as well as those you might expect such as applied demographic research, flood risk monitoring and so on.

Finally, let’s consider how environment is understood in the REF exercise. This section isn’t a list of prizes, awards and ‘peer esteem’ factors. Rather environment is assessed on two criteria: vitality and sustainability. I’ll read you out the REF definitions as they underline my point about thinking differently about research excellence:

“Vitality will be understood as the extent to which a unit supports a thriving and inclusive research culture for all staff and research students, that is based on a clearly articulated strategy for research and enabling its impact, is engaged with the national and international research and user communities and is able to attract excellent postgraduate and postdoctoral researchers.

“Sustainability will be understood as the extent to which the research environment ensures the future health, diversity, wellbeing and wider contribution of the unit and the discipline(s), including investment in people and in infrastructure”.

For my third example, let me touch briefly on changes in my own university to bring this closer to home. One of the key tensions in universities is how the changes I have discussed relate to ideas of career progression and development. Despite what I have said the standards of disciplinary judgement are still seen as significant and powerful, even if such judgements are increasingly out of touch with contemporary complex research practices. Disciplinary judgements focused on research outputs are also inadequate mechanisms to understand the ‘impact’ of research activities, and the forms of experimentation that are resulting in changed behaviours, practices and policies.

Victoria University of Wellington explicitly positions itself as a global-civic university. In this context not long after my arrival, we developed a new Academic Career Framework that built on an enhanced set of indicators for research excellence and also explicitly identified external engagement as an aspect of research activity. My ambition in leading the development of this new Framework was to ensure that my colleagues could see themselves in the list of indicators. Like all universities we are also rethinking our research infrastructure, recognising that as we embed new understandings of engagement, impact and advancement as they will make new demands of our colleagues.

My point in briefly discussing these three areas of work is to underline the changing conceptions of research excellence emerging in an engaged, interdisciplinary and cross-sectoral world. Let me make some further observations about the changes we are seeing as a result of this changed research landscape.

First, it is changing how research problems are identified. In research conducted on this topic in my former university we found that none of the academics who identified as engaged researchers invoked their discipline as the primary arbiter of what areas of inquiry should be pursued. For them defining problem areas, topics of scholarship, and new areas to work in was driven by a much more complex set of factors that emerged out of their interdisciplinary and cross-sectoral engagements. Indeed in a ‘grand challenge’ research world disciplinarity is in many ways a barrier, in that it builds walls between potentially related but currently divided areas of research and practice.

Second, this more holistic conception of research excellence requires different skill sets to those we have traditionally privileged. It challenges the narrow sensibilities of accepted research practice, and privileges collaboration and community building. Established relationships, inter-personal skills, and significant amounts of emotional labour are used to negotiate not just with research partners, but with the rapidly changing expectations of funders and host organisations who now see research collaboration as a means of delivering on wider ambitions. If we need researchers capable of doing this kind of work then we will need to think differently about doctoral training, appointments processes, and academic professional development.

Third, we need to recognise and value a whole set of new actors who are leveraging their knowledge of both traditional research activities and other sectors in this new environment. Consultants, science communicators, public engagement experts, journalists, social entrepreneurs and activists might all play roles in producing and circulating the mobile and mutating knowledges of engaged research. Examples from a co-produced research programme I was involved in include the community activist with a PhD who was contracted to build relationships between three universities and seven community organisations, the graduate student who was also a documentary film-maker and visually captured the collaborative research process, the artist-in-residence who worked to represent traditional research outputs in new ways, and the social enterprise leader who ran their own consultancy and was contracted in to do part time teaching on a community development course.

Fourth, this new environment is changing our funding arrangements. We know from Māori colleagues and community organisations the importance of research partners being named as full co-investigators rather than simply as sub-contractors. This means such partners can be brought in at the beginning of the research process, not simply tagged on a few days before submission, and they are recognised as full contributors to the research process. To deliver more fully on these ambitions for collaborative research we are likely to see the rise of two-stage funding models that allow for the explicit funding of relationship building; and partnerships for in which non-traditional research organisations have control over their own proportion of the funding,

Fifth, a plurality of research outputs emerge from such collaborative research programmes including not just conventional research outputs but also creative works and practitioner resources. This is a particular challenge for universities as non-traditional outputs spread beyond their natural homes in the arts, and began to find their ways into other disciplinary homes. Who else is having the discussion about the formats in which PhDs can be submitted? I certainly didn’t expect to find myself supervising a documentary film, rather than the standard 350 page tome. The research programme I referred to above also produced various art installations, a novel and a wide range of practitioner publications.

There are some final points I want to make by way of a conclusion.

In this new era there is an onus on research organisations to act as guarantor of relationships of care and responsibility towards partners, of research standards, and of financial and legal probity. This might be operationalised, for example, in principles to guide the conduct of engaged and collaborative research, clear statements about the principles that guide academic inquiry and that handle issues of potential conflicts of interest, systems for handling complaints, and financial and legal models that are clear, transparent and meaningful to all parties in a research collaboration. In our increasingly heterogenous research world we need a clear articulation of the principles research should be governed by. This is why Codes and Charters are becoming increasingly important.

Research organisations will need to invest in long term collaborations. Too often our research collaboration models are premised on the continuing interests of an individual researcher and/or short term project funding. Both of these situations can leave community collaborators in particular (industrial partners have fewer such dependencies) in a vulnerable position in the relationship. The challenge is to either exit well once the project funding ends, or perhaps develop a set of shared institutional interests that would be held in common and enable joint investment over a longer period beyond the individual researcher or project.

Our research organisations will also need to facilitate multi-disciplinarity. This isn’t simply about building interdisciplinary networks amongst researchers, although that is hugely important and shouldn’t be under-estimated. It is about actively creating opportunities for stakeholder relationships to be extended beyond initial, often disciplinary, points of contact to create connections across the disciplines. Creating conditions of serendipity, for academics, their collaborators and others to come together, share ideas, create new and surprising connections, is critical. So too is a more structured orientation that actively seeks to encourage this shared project development beyond initial points of collaboration.

Finally, we need to take a meta-level perspective on the collaborations that are developing, and critically examining the types of groups who are systematically over- and under-represented in these processes. Individual researchers will tend to build strong research relationships with those with whom they have shared interests and common backgrounds, or with those who provide access to powerful resources or to specific communities. There is a risk that such an approach leads to a concentration of research energy around specific groups and concerns. There is a critical role for research organisations to play in: first, understanding the nature of the ‘publics’ that are being constructed by the move to engaged research; and second, taking informed decisions about the oversights and exclusions that will necessarily emerge. Such a perspective involves starting from the recognition that, just as research is a highly diverse and contested set of practices, so too are our potential partners diverse, contested, freighted with inequalities and hierarchies of knowledge and resource.

None of this will be straight-forward and much of it is challenging for funders, research organisations and researchers alike. However what should be very clear by now is my key point. In an era of engagement, impact and advancement we will need to think very differently about research excellence. If we do, then our relevance as researchers to the communities we serve and that you represent will be strengthened.

Nō reira,
Whāia te iti kahurangi
Pursue excellence

Kia ora koutou katoa.

This address was given at Te Whare Apārangi in Te Whanga-nui-a-Tara Wellington on Tuesday 30 July 2019.