Explore as a

Share our content

Guidelines for the advisory committee

Guidelines for the advisory committee for the 2019/20 selection round

Advisory-Committee-Guidelines-2019.pdf

General information for Advisory Committee members

This document contains general information about the Centres of Research Excellence (CoREs) Fund, what the expectations are for the conduct of individual Advisory Committee members, and details of the decision making process.  These guidelines MUST be read in conjunction with the Selection Framework.  These guidelines are intended to facilitate the smooth operation of the CoRE Fund Committee meetings.  They are retained as a permanent record, as required by the Auditor-General, and are publicly available.  There are separate guidelines for the Expert Selection Panels and for Referees who also participate in the selection process.

Background to the Centres of Research Excellence (CoREs) Fund

The concept of the CoREs fund was developed following a 2001 review of the tertiary education system by the Tertiary Education Advisory Commission (TEAC). The Commission believed that:

…there was a need for greater concentration of research effort, as well as improved linkages between tertiary providers, industry, other research users and the wider community.

(TEAC 2001, p.103)

The Commission drew on international evidence that research is more likely to be successful (in terms of quality, relevance and impact) if there is a critical mass of researchers who work together to share skills, knowledge and resources.

With the CoREs fund, the Government seeks to address fragmentation across the tertiary education, research, and science and innovation systems to create inter-institutional networks of high-performing researchers.

The CoREs fund is designed to provide incentives for tertiary education research to be undertaken that is outcomes focussed and excellent, and for significant knowledge transfer activities to occur. This combination (outcomes focus, excellence, and knowledge transfer) is also intended to provide incentives for new opportunities for tertiary education research.

Each CoRE is hosted by a tertiary education institution and comprises a number of partner organisations which can include universities, Crown Research Institutes, private research organisations, institutes of technology and polytechnics, and Wānanga. Most CoREs traditionally have close working connections within their wider community of interest.

2019/20 CoREs fund selection process

The 2019/20 Centres of Research Excellence Fund selection round will be a fully contestable round.  This means the round will be open to prospective Centres of Research Excellence; with existing Centres of Research Excellence needing to re-apply and compete for this funding.  Proposals will be considered from all fields of research and Centres of Research Excellence determine their own strategic research direction. 

There are three phases to the assessment of a Full Application as outlined in Figure 1. 

Firstly, applications are sent to referees who comment on the quality of the proposed research programme, and whether it is of a world-class standard. 

Secondly, each application is scored and discussed by an Expert Selection Panel on the basis of research excellence and the contribution of the proposed CoRE within the tertiary education system. 

Finally, the Advisory Committee evaluates the proposed CoRE’s contribution to New Zealand’s future development, and the proposed governance and management structures for the CoRE.  The Advisory Committee, following consultation with the Chairs of the Expert Selection Panels, will decide which applicants will receive a visit to the host site or an interview with the Advisory Committee.  The Advisory Committee will then include the findings of the site visit or interview in their final recommendation to the TEC Board regarding which ten CoREs best meet the selection criteria.

 

Figure 1

Figure 1. A flow chart outlining the process and decision points in the CoREs Fund selection round 2019/20. “Selection” refers to the Expert Selection Panels and “Advisory” refers to the Advisory Committee.

Indicative timetable

Date

Activity

Late Jul 2019

Advisory Committee Chair announced. Open call for nominations for members of the Advisory Committee and Expert Selection Panel chairs.

Late Jul 2019

Selection framework and application guidelines released.

Late Jul 2019

Eligible applicants invited to submit Expressions of Interest.

27 Aug 2019

Expressions of Interest close at 12.00 p.m. NZST (midday), Full Proposal round opens.

Mid Sep 2019

Advisory Committee membership and Expert Selection Panel Chairs announced.

Late Oct 2019

Expert Selection Panel membership announced.

28 Nov 2019

Full applications close at 12.00 p.m. NZDT (midday).

30 Jan 2020

Referees’ reports available from web portal (for comment).

13 Feb 2020

Applicant response period ends.

Mid Mar 2020

Expert Selection Panel meetings to form a long list.

Early Apr 2020

Advisory Committee meeting to form a short list.

Mid-Apr 2020

Short listed applicants advised of site visit schedule.

Late Apr-Early May 2020

Site visits by or interviews with the Advisory Committee.

Mid-late Jun 2020

Recommendations passed to the TEC.

Table 1.         Timetable for CoREs Fund selection round 2019/20

CoREs Advisory Committee

The CoREs Advisory Committee will act as the overall assessment committee and make the final recommendation on the selection of ten CoREs to the TEC.

The members of the Advisory Committee will be people of standing and have the respect of the New Zealand community. They will have high level skills and capabilities, with an understanding of the importance of research to New Zealand’s social, economic or environmental success. It is anticipated that there will be five to seven members of the Committee.

The contribution of the proposed CoREs to New Zealand’s future development and their governance and management structures will be a key focus for the Committee. Assessing these two criteria will be the Advisory Committee’s main responsibility, scored with the following weightings:

  • Contribution to New Zealand’s future development: 20 points
  • Governance and management: 20 points

For further information on the assessment indicators that will be considered during scoring, please see Appendix III– Assessment indicators and scoring framework.

The Advisory Committee will have access to all information on the proposed CoREs.  This will include the Full Application, the referees’ reports and applicant rebuttals, and the rankings of the Expert Selection Panels.  The Committee will focus on the long list of the top  applications as judged by the Expert Selection Panels.

The Advisory Committee will meet in early-April 2020 to discuss the applications.Scoring will be completed individually by Committee members prior to meeting.  Initial average scores for each CoRE are considered a starting point for discussion.  The Committee members will meet to discuss the applications, and the Chairs of each Expert Selection Panel will present the consensus view of their Panel on each proposal with particular emphasis on the recommended long list.  During this meeting the Advisory Committee will reach consensus on a short list of the highest ranked applications.  The Committee members are able to amend their initial rankings in the light of the discussions.  The Advisory Committee will complete the assessment process with site visits to the host institutions or an interview.  In making their final recommendations, the Advisory Committee will take into account all information at their disposal: the applications, the referees’ reports, the applicant responses, the assessments of the Expert Selection Panels on research excellence and contribution within the tertiary sector criteria, their own assessments on contribution of the proposed CoREs to New Zealand’s future development and their governance and management criteria, and information gathered from the site visits or interviews.  The Advisory Committee will then recommend to the TEC which ten proposals it considers best meet the selection criteria. The TEC Board will make final recommendations to the Minister, who will make the final funding decisions.

The Chair of the Advisory Committee is responsible for the effective conduct of this part of the assessment process and ensures that the ranking of proposals is defensible, by:

  • ensuring that the selection criteria are adhered to;
  • ensuring that Committee members consider only the information that has been provided to them through the assessment process;
  • identifying and taking appropriate action over conflicts of interest; and

For further information about the Advisory Committee meetings, please see the “Procedures for the Advisory Committee meeting” section of this document.

Expert Selection Panels

A number of Expert Selection Panels will be established to assess the range of applications.  Applications will be assigned to them by the Society on the basis of the Expressions of Interest.

These panels will be composed of experts in the research fields of interest and have a high level of expertise in the assessment of quality and excellence so that applications can be evaluated and ranked.  Due to the small size of the New Zealand research community, the likelihood of conflicts of interest, and the objective of world-class standard research, panels are likely to include international membership.  The Chairs of the panels will be people of standing and have the respect of the New Zealand research community.  They will have high level skills and capabilities, with a strong background in a relevant research discipline and experience of the New Zealand tertiary education system.

The panels will take into account the referees’ reports and applicant responses, and will consider the applications against the criteria of research excellence and contribution within the tertiary education system.  They will score the applications with the following weightings:

  • Research excellence: 40 points
  • Contribution within the tertiary education system: 20 points

For further information on the assessment indicators that will be considered during scoring, please see Appendix III– Assessment indicators and scoring framework.

The panels will meet in Mid-March 2020 to discuss the applications and will be able to amend their rankings in the light of these discussions.  A panel may decide that no applications assessed by their panel reach the criteria of research excellence.  The panels will recommend the applications that best meet the criteria to the CoREs Advisory Committee and will provide commentary on all applications for the TEC. All applications recommended by the panels will be combined into a long list for assessment by the Advisory Committee.  The panel Chairs will present the consensus views of their panel on each proposal that they have assessed at a meeting of the Advisory Committee in early-April 2020, with particular emphasis on those applications that have been long listed for recommendation.

Referees

All applications will first be assessed by referees who will comment on the proposed research programme of the CoRE with respect to research excellence, its research team, and its intended methodology for research and knowledge transfer.  Referees will receive the full application, but will be asked to focus their attention on sections 1-8 (see Application Guidelines for details).

Referees will be internationally recognised experts in the relevant fields of research. Their comments will be an important element of the assessment of whether a proposal is for leading edge research of world-class quality.

The Society will seek reports from at least three referees for each Application, although this may not be attainable for a variety of reasons. Applicants will be asked to suggest the names of suitable referees; however, whether a referee is asked for a report will depend on their availability and on an assessment of any conflicts of interest. A list of potential referees for each Application will be developed by the CoREs secretariat in consultation with the relevant Expert Selection Panel. Depending on the nature of the proposal, more than three referees may be contacted.

Applicants will be given the opportunity to provide a two-page response to each referee’s comments. The referees’ reports and the applicants’ responses will form part of the material for Expert Selection Panels’ and the Advisory Committee’s assessment of the applications.

Royal Society Te Apārangi Staff

It is not the role of the Royal Society Te Apārangi to make funding decisions.  Rather, their role is one of facilitation and “guardianship” of the assessment process, ensuring that the process is credible and defensible.  To achieve this, staff will:

  • organise all logistical aspects of the process;
  • assist the Chair of the CoREs Advisory Committee in determining realistic timetables for meetings and visits;
  • record decisions of Expert Selection Panels and Advisory Committee meetings;
  • record any conflicts of interest and actions taken; and
  • forward the final recommendations to the Tertiary Education Commission.

Recommendations to the TEC

TEC has asked for recommendations for ten CoREs, with at least one of them being focussed on Māori research. Funding of $49.8 million per year is available for the CoREs. The 2019/20 CoREs selection round is for operating funding only, and is a fully contestable round.

The CoREs Advisory Committee will recommend to the TEC which proposals it considers should be funded and provide advice on the level of funding to award. The TEC Board will make the final selection recommendations and provide these to the Associate Minister for Education for a final decision to be made.

Feedback to Applicants

All applicants will receive an acknowledgement of the receipt of the Expression of Interest and the Application. The Society can provide information and guidance for potential applicants until the closing date for applications. Applicants will receive copies of the comments made about their proposals by the referees, but otherwise will not receive any direct communication from the Society about the outcome of the assessment. The rationale for the decisions made during the selection process leading to the recommendations for funding will be provided to TEC.

Privacy

The Royal Society Te Apārangi has obligations under the Privacy Act to keep confidential certain information provided by individuals.  Moreover, the records of deliberations by the Advisory Committee are regarded as strictly confidential, as are the contents of unsuccessful applications.

  • Advisory Committee members must ensure the safe keeping of all applications and related confidential documents (e.g. application spreadsheets, referee reports, meeting summaries).
  • At the conclusion of the Advisory Committee meetings, members must leave documentation with the Royal Society staff and destroy any documentation remaining elsewhere.
  • Outside the meetings of the Advisory Committee there must be no correspondence or discussion by Committee members of the contents of the applications with referees, third parties, or the applicants, except by the Chair of the Committee or the Society’s designated CoREs secretariat.
  • The intellectual property of the ideas and hypotheses put forward in the applications must be treated by the Advisory Committee in strict confidence.

Conflicts of Interest

The Royal Society Te Apārangi takes the issue of conflict of interest very seriously. A rigorous position must be taken in order to maintain the credibility of the allocation process and to ensure that applications are subjected to fair and reasonable appraisal.  During Committee member selection the Society will try, as far as possible, to minimise the known conflicts of interest in the Committee.  However, where further conflicts of interest arise for Committee members the following rules will apply.

  • A Committee member may not be part of an application to the Fund.
  • If a Committee member has an interest in an application, such as collaborating with an applicant or an applicants’ group, or being a close relative, then that member, at the discretion of the Advisory Committee Chair, shall leave the room, remain silent or answer technical questions only.
  • If the Advisory Committee Chair has a conflict of interest then the duties of chairing the meeting shall be passed to a deputy.
  • All the above conflicts of interest must be declared in writing to the Society.
  • The CoREs secretariat will ensure minutes are recorded of all conflicts of interest and actions taken.

Procedures for the Advisory Committee Meetings

There are expected to be three meetings of the Advisory Committee: the first in early December to familiarise the members with the framework and processes involved in assessing the proposed and to discuss how the Advisory Committee will work within the framework of the guidelines and selection criteria; the second in late February to discuss the long list applications recommended by the Expert Selection Panels and create a short list for site visits or interviews; the Advisory committee will conduct site visits or interviews in late April; and the third in May, following the site visits or interviews, to finalise the recommendations for funding to the TEC.

Decision Making in the Advisory Committee Meetings

The CoREs Advisory Committee will work on the basis of consensus based decisions, rather than voting.  Where a decision is required the Chairman will, after reasonable deliberation and when the Chair judges that there is a consensus amongst the Committee members, state that: the Chair believes that agreement has been reached; and what the agreement is.

If any member of the Committee considers that a consensus has not been reached they may say so and discussion will continue, including on whether there is a consensus.  There may also be discussion on exactly what the agreement entails.

This means that each Committee member fully supports the outcome of the Committee’s deliberations but cannot be assumed to have supported every element of every decision.  Therefore, they do not need to state dissenting views on particular items that they did not fully agree with.

Grading and Ranking

The scoring system for the 2019/20 selection round is as follows:

Criterion

Expert Selection Panel

Advisory Committee

Total

Research Excellence

40

 

40

Tertiary education system

20

 

20

NZ’s future development

 

20

20

Governance & management

 

20

20

Total

60

40

100

The CoREs secretariat will review the scoring consistency of individual members of the Expert Selection Panels. Where extremes in scoring behaviour are identified, it will be brought to the attention of the relevant ESP Chair.

Scoring will be completed individually by the Expert Selection Panel members and the Advisory Committee members prior to meeting. Initial average scores for each proposed CoRE are considered a starting point for discussion and rankings after each meeting will reflect the consensus view of the relevant panel or of the Committee.

Following the site visits or interviews, the Advisory Committee will make recommendations on funding to the TEC. These recommendations will be based on a holistic overview of all information at the Committee’s disposal (the applications, the referees’ reports, the applicant responses, the views of the Expert Selection Panels on research excellence and contribution within the tertiary sector, their own views on contribution of the proposed CoREs to New Zealand’s future development and their governance and management, and information gathered from the site visits or interviews). Final recommendations will reflect the consensus view of the Committee following discussion.

December Meeting

This meeting, occurring via videoconference, is intended to provide the Advisory Committee members with the chance to meet each other, and to get an overview of the CoREs selection process and the funding environment within New Zealand at the moment.  It is also intended to allow the Advisory Committee to discuss how they will work within the framework of the guidelines and the selection criteria with a view to their decision making process in the April meeting.

April Meeting

The purpose of this meeting is to review applications on the long list provided by the Expert Selection Panels and to generate a short list of proposed CoREs for the Advisory Committee to a site visit or interview.

The Committee members will receive copies of all the applications, the referees’ comments and the applicants’ rebuttals on after 13th February 2020 in order to familiarise themselves with these documents.  Members will be asked to identify proposals for which they have a conflict of interest, explaining the nature of the conflict.

Following the Expert Selection Panel meetings in Mid-March, the Advisory Committee members will be advised in late March of which applications recommended by the Expert Selection Panels as the long list.  They will also receive the Expert Selection Panels’ comments on these proposed CoREs. The recommended applications will be assessed by all members unless a conflict of interest has been declared.

Committee members will be provided with a spreadsheet listing the applications with room for initial grades and comments to be entered, to be sent to the CoRE Fund administration.  This will enable proposals to be given an initial overall ranking for discussion at the meeting. 

At the Committee meeting, a summary sheet will be provided to each member recording the initial overall rankings generated from the members’ grades for each application and the scores given by the Referees and Assessment Panels.  Any changes in rankings subsequently made during the meeting will be recorded.  The Advisory Committee will also discuss the wording of any feedback information for the applicants.

The Advisory Committee will have available all the information from the assessment panels, the referees and the applicants responses to referees.  The Committee will choose proposals as a short list for site visits by or interviews with the Advisory Committee.

April/May Site Visits or Interviews

Following the April meeting, the Advisory Committee will conduct site visits to each host institution, or interviews of applicants of the short listed proposed CoREs.  These site visits or interviews will allow members of the Advisory Committee to ask further questions and raise issues that are not readily addressed in the written proposal.  The visits also allow the Committee to assess the suitability of the host organisation’s provision of facilities, and to observe interactions between representatives of both host and partner organisations.  Each site visit or interview is anticipated to last for approximately half a day.

May Meeting

Immediately after the site visits or interviews have been conducted, the Advisory Committee will reconvene to discuss the short list of proposed CoREs in light of all the information now at their disposal.  They will then make their final recommendations of ten CoREs to receive funding to the TEC.

Appendix I – CoREs Mission Statement

Government’s investment intention

It is the intention of the Government that investing in Centres of Research Excellence (CoREs) will support growth in research excellence and the development of world class researchers in areas of existing excellence that are important to New Zealand’s future development.

Mission statement for CoREs

This mission statement sets out high level expectations of CoREs in terms of role, performance and achievements, as well as the shared responsibilities of host and partners.

The mission statement forms the basis of selection criteria and ongoing performance monitoring of the CoREs.

Research carried out by CoREs

CoRE research must be leading edge research of world-class quality in an area of importance to New Zealand. CoRE research demonstrates academic strength as well as planned and effective progress towards defined impacts with public good and/or economic benefits.

  • CoRE research may be basic and/or applied. It is always pioneering, commonly multi-dimensional and/or multi-disciplinary, and likely to involve collaborative and inter-institutional participation and exchange.
  • A CoRE will define its area of strategic impact, the need for this focus and the potential benefit for New Zealand. It will regularly revisit and refresh research planning to ensure its research is innovative and solution-focussed.
  • A CoRE is innovative and responds quickly to opportunity. It allocates funding for excellent research that has next-stage potential impact and is aligned with its strategic direction.
  • It is anticipated that, over time, the research and personnel profile of a CoRE will evolve within its area of strategic impact, reflecting the innovative and cutting edge nature of a CoRE’s research activities.
  • A CoRE commits to equity and wellbeing outcomes, including encouraging and enabling diversity and inclusion in its research activities and/or its research team(s).
  • CoRE expertise can be directed to government science priorities, as appropriate.

CoREs as an element in the tertiary education system

A CoRE builds research capability in areas of existing excellence. It translates new knowledge into teaching and the training of future researchers and offers specific and novel opportunities for graduate students, emerging and established investigators, across its partner institutions.

  • A CoRE plans its contribution to the teaching and learning environment of its partner institutions. It is able to demonstrate its educational outcomes and its contribution to employment outcomes for graduates.
  • A CoRE commits to equity and wellbeing outcomes, including encouraging and enabling diversity and inclusion in its teaching activities.
  • A CoRE ensures key investigators have the opportunity to influence the experience of graduate students and postdoctoral fellows working with the CoRE.

CoRE collaboration and collaborative practices within CoREs

The CoRE model is a collaborative research partnership hosted by a TEI. It has appropriate governance and processes to ensure all partners contribute to delivery of agreed strategic outcomes.

  • The host and partners share responsibility for the development and regular review of institutional partner agreements.
  • Agreements recognise a combined responsibility for resourcing the CoRE as well as ensuring that research is of excellent quality and adds strategic benefit.
  • Agreements set out host university and partner contributions, which include financial and/or in-kind contributions.
  • The CoRE host and partners together agree and implement collaborative policies and practices.

Engagement by CoREs with end-users and stakeholders

CoREs are characterised by active and outward-facing engagement with next-stage and potential end-users to ensure the CoRE delivers the strategic benefits it seeks for New Zealand.

  • A CoRE will invite potential end-users to contribute to planning of current and future research options, including consideration of potential for impact.
  • A CoRE engages in research translation to support and encourage research uptake.
  • CoRE outreach can take many forms and have many audiences. A CoRE will identify its outreach partners and explore with them the best ways to engage.
  • A CoRE develops an engagement plan to bring focus to its dialogue with stakeholders, and its knowledge exchange activities and connections.

The role of the CoRE at a national and international level

A CoRE demonstrates authority in its research area, both in New Zealand and overseas.

  • A CoRE builds wide networks within national and international research communities and uses this connectivity to strengthen its research, people, engagement and influence.
  • A CoRE deploys its leadership to facilitate wide stakeholder debate on issues of significance.
  • A CoRE operates as a showcase for New Zealand.

Appendix II – Selection Criteria

Prospective CoREs will be assessed on the basis of four selection criteria. The assessment criteria are drawn from the Minister of Education’s letter of determination issued to the TEC under 159L of the Education Act issued in May 2019. The selection criteria must be read in conjunction with the CoREs Mission Statement in Appendix I.

Excellence: excellent research, including assessment of:

  • academic strength of the proposed research programme;
  • academic strength of the proposed research team;
  • commitment to ensuring research is innovative, outcomes-focussed and has impact;
  • strength of proposed collaboration and the degree to which partners have contributed to the proposal;
  • potential of the CoRE to have national and international influence; and
  • commitment to equity and wellbeing outcomes, including encouraging and enabling diversity and inclusion for Māori, Pacific peoples, and other under-represented groups in the proposed research programme and/or in the proposed research team.

Contribution within the tertiary education system, including assessment of:

  • contribution to the priorities of the Tertiary Education Strategy;
  • contribution to graduate, and postgraduate and new researcher education;
  • expected impact on the development of New Zealand’s future workforce;
  • contribution to the development of a culture of innovation and wealth creation in New Zealand; and
  • commitment to prioritising equity and wellbeing outcomes, including encouraging and enabling diversity and inclusion for Māori, Pacific peoples, and other under-represented groups in teaching activities.

Contribution to New Zealand’s future development, including assessment of:

  • potential for the research to have public good and/or economic impact in New Zealand;
  • commitment to engagement and exchange with potential stakeholders/end-users; and
  • strength of the intention to drive the connections that lead to research translation.

Governance and management, including assessment of:

  • strength of planned governance and management;
  • clarity and benefits of proposed host and partner contributions; and
  • commitment of parties to proposed collaborative practices.

In addition, the selection process will consider how the CoREs will support the goals set out in:

When selecting CoREs to be funded, CoREs that best meet the assessment criteria will be selected.

At least one CoRE focussed on Māori research will be funded through a competitive process in accordance with the assessment criteria. If no proposals meet the selection criteria, the Advisory Committee may recommend the proposal that best meets the criteria and the TEC will work with the applicant to ensure that the proposal is further developed and the resulting CoRE is of a comparable standard to other CoREs.

Appendix III – Assessment indicators and scoring framework

Assessment indicators

The applications will be assessed and scored against the four selection criteria.

Research excellence:

The excellence of the research programme and the commitment to ensuring research is leading edge and outcomes focussed will be determined by assessing: whether the research proposal is well constructed and represents a creative and pioneering approach to the subject matter; it is likely that significant new findings will be produced; it will substantially advance knowledge in the field; the proposed collaborations are likely to spark new directions in the research; the programme intends to develop new techniques or technologies that can be applied to relevant problems; and the Centre has the ability to respond quickly to opportunity.

The academic strength of the research team together with the breadth and depth of their knowledge and experience will be assessed by reviewing: the previous research productivity of the Principal and Associate Investigators including their publication record; key presentations; the structure of the research group for future development (incorporating senior and less experienced researchers, postdoctoral fellows, postgraduate students and appropriate other assistance); national/international awards to the research leaders for achievements relating to their research; the ability to attract new researchers to the field; and the credentials of the Centre’s Director, such as academic and professional standing, and their capacity for research leadership.

The collaborative depth of the proposed partnership will be evaluated by reviewing: the significance of the additional skills and resources; the inclusion of new researchers; the level of exchange of visitors and research personnel; and the extent of participation in international events.

The proposed national and international influence of the CoRE will be determined by: the international standing of the research team; the likely impact of the proposed research within the wider field; the likely impact of the research on questions and goals specific to New Zealand; and the ability of the CoRE to facilitate wide stakeholder debate on issues of significance.

Applicants should consider the following questions:

  • Is the academic strength of the proposed research programme demonstrated?
  • Is the academic strength of the proposed research team demonstrated?
  • Is there a commitment to ensuring research is innovative, outcomes-focussed and has impact?
  • Does the proposed collaboration and the degree to which partners have contributed to the proposal indicate that there is meaningful collaboration and investment in the proposal?
  • Is the CoRE likely to have both national and international influence?

Is there a commitment to equity and wellbeing outcomes, including encouraging and enabling diversity and inclusion in the proposed research programme and/or in the proposed research team?

Contribution within the tertiary education system:

The Tertiary Education Strategy (TES) 2014-19 highlights the need to build international relationships that contribute to improved competitiveness, to support business and innovation through development of relevant skills and research, and to improve outcomes for all. The TES has a strong focus on improving economic outcomes from tertiary education and research and to improving environmental and social outcomes.

The ability of the Centre to contribute to the tertiary education system will be assessed on their strategies to build human capability and the workforce required to participate in a global economy. 

The contribution the Centre will make to educating and training New Zealand’s future workforce will be evaluated by the extent to which the programme contributes to the successful completion of degree and postgraduate qualifications, including increased achievement of Māori and Pasifika; teaching and learning environment of its partner institutions; and encourages the development of research skills, with the key investigators able to influence the experience of graduate and postgraduate students and new researchers working with the CoRE. The Centre is expected to have research leaders with strong research supervision records; and a programme that is linked to the skill needs of the relevant end-user community.

It is expected that a successful Centre will contribute to a culture of innovation and wealth creation in New Zealand, and that it will develop strategies and monitoring systems to measure its progress in contributing to innovative activity.

Applicants should consider the following questions:

  • Does the proposal make a significant contribution to the priorities of the Tertiary Education Strategy 2014-2019?
  • What is the CoRE’s contribution to graduate, postgraduate and new researcher education? Is the proposal likely to have a strong impact on the development of New Zealand’s future workforce?
  • Does the proposal demonstrate contribution to the development of a culture of innovation and wealth creation in New Zealand?
  • Is there a commitment to prioritising equity and wellbeing outcomes, including encouraging and enabling diversity and inclusion in teaching activities?
  • Is there a connection to undergraduate and compulsory-sector initiatives to broaden participation in post-graduate research?

Contribution to New Zealand’s future development:

The ability of the applicants to enhance New Zealand’s future development will be assessed by: the potential for the research to provide for public good and/or economic impact in New Zealand; the extent to which addressing these goals is a focus of the Centre; the extent to which the proposed Centre duplicates or expands existing capabilities within New Zealand’s research system; and the extent to which new Centres describe their potential to generate, and existing Centres plan to maintain, their value through capability and network development.

The commitment to engagement and exchange with potential stakeholders/end-users will be evaluated by: the understanding shown of the research needs of New Zealand communities and industries; plans for effective translation of research and knowledge into application for new and existing commercial success and/or social or environmental benefits; involvement of relevant sectors in determining the Centre’s research programme; and the demonstration of innovative value from the new or expanded activity in service or policy development, or in enhancing productivity, developing new products and helping New Zealand firms compete in high-value products and services.

The strength of intention to drive the connections that lead to research translation will be assessed by: the relationship to other groups in the particular field of research; the involvement of end-users and the wider community in the planning, implementation and uptake of the research programme; the plans for promoting the Centre’s activities to the wider community, including where appropriate, for commercial gain; the strength of institutional connections with stakeholders in industry, community, and iwi; and the impact on Māori and Pasifika from the research.

In addition to explaining why the research is important to New Zealand, applicants should consider the following questions:

  • Is there potential for the research to have public good and/or economic impact in New Zealand?
  • Is there a commitment to engagement and exchange with potential stakeholders/ end-users?
  • Is there a strong intention to drive the connections that lead to research translation?

Governance and management:

The strength of the governance, management and operational arrangements will be evaluated by: considering the Centre’s Charter or Business Plan and proposed performance measures to determine the clarity of the principles of operation and the extent to which the governance and management structures will ensure these are met; the credentials of the Centre’s Director such as capacity for strategic leadership, management skills and successful experience; a robust Centre establishment plan; the adequacy of the proposed reporting arrangements; the ability for the Centre to meet its primary purpose; the ability to fulfil Treaty of Waitangi obligations; host institution support, such as Centre establishment, funding or other financial policies, provision of space and other resources; and the financial soundness of the proposed Centre.

The clarity and benefits of proposed host and partner contributions will be evaluated by considering any Memoranda of Understanding or other agreements that those organisations have put in place, including: agreements regarding combined responsibilities for resourcing the CoRE; and agreements setting out host and partner contributions including financial and/or in-kind contributions (e.g. agreements for personnel sharing, cost sharing, and intellectual property sharing).

The appropriateness of the Centre’s facilities will be assessed by: the ability for the centre to develop a separate identity from its host; the standard of the accommodation; the extent to which the equipment and other research tools are considered state of the art; and the standard and ease of access to support services, including financial systems provided by the host.

The commitment of parties to proposed collaborative practices will be determined by agreements recognising the combined responsibility of host and partners for ensuring that research is of excellent quality and adds strategic benefit.

Applicants should consider the following questions:

  • Does the planned governance and management show strength and an ability to deliver on the CoRE’s impacts?
  • Is there clarity around proposed host and partner contributions, and are the benefits clear?
  • Have the partners committed to strong collaborative practices?

Scoring framework

Score weighting

Criterion

Expert Selection Panel

Advisory Committee

Total

Research Excellence

40

 

40

Tertiary education system

20

 

20

NZ’s future development

 

20

20

Governance & management

 

20

20

Total

60

40

100

Scoring guide

Assessors will use the following to guide scoring. Assessment is judgement-based, and assessors will apply their knowledge and experience to interpret the information and evidence in each application and provide a score for each criterion. This will form the basis of a holistic ranking by panels and the committee.

High

Research: 31-40 pts

Remaining criteria: 16-20 pts

The application meets the criterion to a high degree. The evidence:

  • provides a strong indication that the Centre would meet most, if not all, of the elements of the criterion, and
  • is strong and credible.

Moderate

Research: 21-30 pts

Remaining criteria: 11-15 pts

The application meets the criterion to a moderate degree. The evidence:

  • provides an indication that most, but not all of the elements of the criterion are met, and/or
  • is not consistently strong and/or credible

Low

Research: 11-20 pts

Remaining criteria: 6-10 pts

The application meets the criterion to a low degree. The evidence:

  • provides an indication that some, but not most, of the elements of the criterion are met, and/or
  • is not consistently strong and/or credible.

Inadequate

Research: 0-10 pts

Remaining criteria: 0-5 pts

The evidence provided does not meet the criteria for Low grade. The evidence:

  • is missing, or
  • is not responsive to the criteria, and/or
  • overall is not strong and /or credible.

Scoring indicators

Indicators and the types of evidence that could aid in assessment of indicators

Criterion

Indicators

Types of evidence that could be presented

Excellent research (weighting 40 points)

Academic strength of the proposed research team

  • Credentials of the Centre’s Director, such as academic and professional standing, and their capacity for research leadership
  • Research quality and productivity of the Principal and Associate Investigators
  • The structure of the research group supports succession and sustainability of resources for future development (incorporating senior and less experienced researchers, postdoctoral fellows, postgraduate students and appropriate other assistance)
  • The ability to attract new researchers to the field
  • CVs
  • Local, national and international awards
  • Investigator publication records and key presentations
  • Information on the composition of the team/organisation charts
  • Information on recruitment performance, e.g. from annual reports
 

Academic strength of the proposed research programme

Whether the research proposal is well constructed and represents a creative and pioneering approach to the subject matter, and the Centre:

  • is likely to produce significant new findings
  • is likely to substantially advance knowledge in the field
  • the proposed collaborations are likely to spark new directions in the research
  • the programme intends to develop new techniques or technologies that can be applied to relevant problems
  • the Centre has the ability to respond quickly to opportunity.
  • The research proposal
  • Evidence that the CoRE or participants have produced significant findings, advanced knowledge, sparked new directions, developed new technologies, and responded quickly to opportunities in the past
 

Commitment to ensuring research is innovative, outcomes-focused and has impact

  • Clear statement in the proposal of commitment to ensuring research is innovative, outcomes-focused and has impact
  • Statement is consistent with and likely to be borne out by the CoRE if operated consistent with the proposal
  • The proposal
 

Strength of proposed collaboration and the degree to which partners have contributed to the proposal

  • The collaborative depth of the proposed partnership will be evaluated by reviewing the significance of the additional skills and resources brought to the research programme.
  • the inclusion of researchers from partner organisations
  • the level of exchange of visitors and research personnel among partners and or collaborators
  • The existence of formal established collaborations and proposed future collaborators and what the nature of the interaction will be.
 

Potential of the CoRE to have national and international influence

  • participation of leaders and staff in international events
  • the international standing of the research team
  • the likely impact of the proposed research within the wider field
  • the likely impact of the research on questions and goals specific to New Zealand
  • the ability of the CoRE to engender discussion among stakeholders on issues of significance.
  • Invitations to participate in international conferences
  • CVs of leaders and staff
  • Agreements, contracts for international exchanges
  • Scope and thrust of the proposal
 

Commitment to equity and wellbeing outcomes, including encouraging and enabling diversity and inclusion for Māori, Pacific peoples, and other under-represented groups in the proposed research programme and/or in the proposed research team

  • Clear statement in the proposal of commitment to well-being outcomes
  • Statement is consistent with and likely to be borne out by the CoRE if operated consistent with the proposal
  • Team composition/ demographic information
  • CVs
  • Team development plans
  • Scope and thrust of application
 

Contribution within the tertiary education system (weighting 20 points)

Contribution to the priorities of the Tertiary Education Strategy

  • Priority 1 – Delivering skills for industry
  • Priority 3 – Boosting achievement for Māori and Pasifika
  • Priority 5 – Strengthening research-based institutions
  • Priority 6 – Growing international linkages
  • Consistency with any evidence provided regarding other criteria, including the strength of the team and the proposal, diversity and inclusion, and national and international influence.
  • The proposal includes factors that encourage development of research skills and specifically respond to the relevant elements of the Tertiary Education Strategy
  • Specific strategies for developing staff and students, and teaching and learning environments
  • Evidence provided regarding Māori and Pasifika, international, strength of team and proposal
  • Research supervision records
  • Engagement with and plans to deliver on the needs of the end-user community
   

Contribution to graduate, and postgraduate and new researcher education

  • Consistency with evidence provided regarding other indicators under this criterion
  • Articulation of contribution in proposal
  • Team design and resource planning
  • Records of communication of opportunities and placements
  • Strategies to ensure a meaningful contribution to graduate, and postgraduate and new researcher education
 

Expected impact on the development of New Zealand’s future workforce

  • Consistency with evidence provided regarding other criteria including contribution to New Zealand’s future development, commitment to equity
  • Clear explanation in proposal about the expected impact
 

Contribution to the development of a culture of innovation and wealth creation in New Zealand

  • Consistency with evidence provided regarding criteria related to contribution to New Zealand’s future development
  • Clear explanation in proposal about how the Centre will contribute
 

Commitment to prioritising equity and wellbeing outcomes, including encouraging and enabling diversity and inclusion for Māori, Pacific peoples, and other under-represented groups in teaching activities

  • Consistency with evidence provided regarding other relevant criteria, including diversity and inclusion
  • Clear statement in the proposal about this commitment
  • Organisation charts
  • Staff turnover information
 

Contribution to New Zealand’s future development (weighting 20 points)

Potential for the research to have public good and/or economic impact in New Zealand

  • The focus of the proposal on delivering public good and economic impacts
  • The extent to which the Centre duplicates or expands existing capabilities within New Zealand’s research system
  • The extent to which new Centres describe their potential to generate, and existing Centres plan to maintain, their value through capability and network development
  • Clear statement in proposal about the potential impacts
 

Commitment to engagement and exchange with potential stakeholders/end-users

  • the understanding shown of the research needs of New Zealand communities and industries
  • Plans for effective translation of research and knowledge into application for new and existing commercial success, and/or social or environmental benefits
  • Involvement of relevant sector in determining the Centre’s research programme
  • Demonstration of innovative value from the new or expanded activity in service or policy development, or in enhancing productivity, developing new products and helping New Zealand firms compete in high-value products and services
  • The proposal
  • Centre strategy and business plan, and specific plans for translation of research, commercialisation
  • Engagement schedules and records of meetings
  • Communications plans and strategies
  • Records of sales or other arrangements that resulted in value being derived
 

Strength of the intention to drive the connections that lead to research translation

  • Relationships with other groups in the particular field of research
  • The involvement of end-users and the wider community in the planning, implementation and uptake of the research programme
  • Plans for promoting the Centre’s activities to the wider community, including where appropriate, for commercial gain
  • The strength of institutional connections with stakeholders in industry, community, and iwi; and the impact on Māori and Pasifika from the research
  • Consistency with evidence provided regarding other criteria, particularly in relation to the impacts of research
  • Clear statement of expectations of impacts in proposal
  • Agreements and plans to work with other groups and engage with end-users and the community
 

Governance and management (weighting 20 points)

Strength of planned governance and management

(ability to deliver on the Centre’s purpose, Treaty of Waitangi obligations)

  • Well-designed governance structure and operational processes
  • Experienced and stable leadership at both levels
  • Policies and procedures framework
  • Board performance expectations
  • Robust resourcing plans, including financial
  • Host institution capacity for space and other resources
  • Organisational charts
  • Chair and CE credentials
  • Board and senior leadership turnover
  • Charters and board policies
  • Board cycles, calendars, agendas
  • Board self- and independent review
  • CE review
  • Strategies and business plans, budgets, performance measures, and reports
  • Establishment plans for new Centres
 

Clarity and benefits of proposed host and partner contributions

  • Ability to establish a separate identity for the centre
  • Quality and fitness for purpose of facilities and equipment
  • Standards and access to support services, including financial systems provided by the host
  • Memoranda of understanding, contracts and other arrangements with partner agencies for sharing resources, including, space, funding, personnel, in-kind contributions, intellectual property
 

Commitment of parties to proposed collaborative practices

  • Agreements recognise the combined responsibility of host and partners for ensuring that research is of excellent quality and adds strategic benefit
  • Clear statement of commitment in proposal
  • Agreements between host and partner organisations
 

The CoREs secretariat will review the scoring consistency of individual members of the Expert Selection Panels. Where extremes in scoring behaviour are identified, it will be brought to the attention of the relevant ESP Chair.

Scoring will be completed individually by the Expert Selection Panel members and the Advisory Committee members prior to meeting. Initial average scores for each proposed CoRE are considered a starting point for discussion and rankings after each meeting will reflect the consensus view of the relevant panel or of the Committee.