Table of Contents

• Assessing the quality of publishing venues, and key issues in academic publishing
• Questions to ask of a potential publisher
• Affiliation and contact information
• Editorial practices
• Peer-review process
• Indexing
• Fees
• Indicators of poor practice
• Indicators of quality
• Other checklists
• Key issues in academic publishing
• The rise of Open Access and pay to publish
• New entrants in the scholarly information environment
• The effect on the scholarly record
• A risk that good research becomes irretrievable
• A risk that poor quality research is published
• Note to users
• Acknowledgements
• Further information and reading

Assessing the quality of publishing venues, and key issues in academic publishing

Guidance from Royal Society Te Apārangi

Getting published is important, but not so important that you should publish in an inappropriate or disreputable journal. This guide provides information to help you navigate the publishing world, with questions to ask of a potential publisher and information on key issues in academic publishing.

Questions to ask of a potential publisher

No freely available, current, curated listing of unethical publishers exists and, in reality, it is extremely difficult to find out if a journal is a genuine, scholarly publishing venue.

The following questions aim to help researchers assess whether a publishing outlet that they are considering as a venue for dissemination of their research is likely to be a bona fide operation, as opposed to a ‘predatory’ publisher. If a journal does not answer positively to a small number of the following questions, this does not necessarily indicate that it is a ‘predatory’ journal and further research is required. Publishers offer a service to scholars and, as for other goods or services, the principle of ‘buyer beware’ should prevail.

Some new journals, run voluntarily by academic staff, may not have the resources to provide the same online readership experience as a well-established journal published by a commercial publisher and their websites may look somewhat ‘unprofessional’. This does not, however, indicate that they are an unethical publisher. English is the common language of science and hence there is a preference to publish in English language journals, but poor quality English on a website is not necessarily an indicator of a poor quality or unethical publisher; the publisher may simply not have the resources to correct the language to native standard.

Before trying to assess the quality of potential publication outlets for their research, researchers should ask their institution’s library to look into their preferred publishing venue. Libraries often have access to directories that list bona fide publications. Researchers should also consult with experienced colleagues if they are unsure of where to send their work for publication.

Note: Guidance on conferences run by commercial enterprises is beyond the scope of this work but scholars are advised to thoroughly research the organiser of any conference if it is unfamiliar to them.

Affiliation and contact information

1. Who ‘owns’ the journal – a scholarly society, a reputable commercial publisher, an educational establishment

This information should be available on the journal’s website.

2. Does the journal’s name match its location (British journal of…)?

Some dubious publishers claim bogus addresses in many countries. A physical contact address and a commercial or institutional email address for the publisher should be available, i.e. not an address from a generic and free email service provider.

Editorial practices

3. Are turnaround times realistic: generally weeks or months, rather than days? 

Turnaround times may include either an average time taken for peer-review and/or time from acceptance to publication, or simply time from submission to publication. Guarantees of overly short times for publication can be a warning that thorough peer-review is not being undertaken. However, be aware that some bona fide publications do offer very fast turnaround times so this is not, in itself, an indicator of a ‘predatory’ publishing operation. The practice of open, post-publication peer-review is significantly speeding up the publishing workflow and can lead to very rapid turnaround times. A transparent, easily accessible peer-review policy for a journal must be available for consultation. Such a policy should state whether the editor carries out ‘desk rejections’ without sending work for peer-review if they view the work to be unsuitable for the journal, or if editors carry out peer-review themselves for any reason.

4. Are the following easy to find on the website?

• a clear editorial and review policy
• aims and scope that are closely related to the title of the journal
• author instructions
• an editorial board made up of experts in the field, preferably with institutional contact email addresses. Ideally, a brief biography of each editor should be available
• Information on article processing charges
• Turnaround times for the various stages of publication
• A list of places where the journal is indexed
  
  

5. Can you retain copyright of your work? If not, what is the copyright policy of the publisher?

Most quality Open Access journals will publish articles under one of the Creative Commons licenses rather than ask authors to assign copyright to the journal. However, this is not always the case, as some reputable journal publishers retain copyright even in an Open Access publication. 

6. Do you know the editors? If so, are they recognised experts in the field? Are their contact emails provided?

Note that the names of some scholars may have been listed on journals’ editorial boards without their knowledge, or that editors who initially agreed to an association with a particular journal have been unable to disassociate themselves once the nature of the publishing operation became clear. If in doubt, contact the journal editors to ask about their work with the journal and check the websites of the institutions of each editor to ensure that they are, in fact, an employee of the organisation.

Peer-review process

7. Does the publication carry out independent, external peer-review of work submitted and, if so, what level of peer-review is carried out?

A basic peer-review may only examine the research methodology; a more thorough review should examine methodology, research question, standard of writing, literature reviewed, presentation and interpretation of results, and any conclusions made. Editorial staff should not carry out the peer-review themselves. A casual check of a journal’s policy may not be revealing as the policy can state a very rigorous process but may not send every paper for review. Best practice is to contact authors whose work has been published to ask about the peer-review process for the journal.

Indexing

8. Is the journal indexed in scholarly databases such as Web of Science or Scopus, or others that you use in your information searches?

Some ‘predatory’ journals may have been indexed in these databases and, although periodic vetting removes poor quality journals from the databases, existing records aren’t removed.

If a journal claims to be indexed in the main databases, carry out a search to check that this information is true.

Fees

9. If the journal is Open Access, how is it funded? 

The most common funding model is through Author Publication Charges, or APCs. Any such fees should be clearly stated on the journal’s website. Sometimes, an Open Access journal may be financially supported by an institution or government.

Indicators of poor practice

10. Does the journal have a regular publishing schedule (weekly, monthly, annually)?

A highly irregular schedule may be of concern, particularly for a suite of journals run by one publisher across a range of disciplines. A one-off journal run by scholars may also have an irregular publishing schedule but this is of less concern, as such journals are often run by volunteers who have less time to commit to editorial work.

11. Does the publisher publish one or two journals in a specific discipline or a large number of journals across a range of disciplines?

Unless the publisher is one of the better-known commercial publishers, a multidisciplinary suite of quality journals is rare.

12. Has there been a sudden surge in articles from a particular region or country or in the number of articles published?

Look at previous journal issues to compare whether the number of articles published in previous years is similar to the current issue’s extent. However, if a field of study is expanding, a journal may increase its extent. Such an increase might be accompanied by an editorial, explaining the reason for the expansion of the journal.

Indicators of quality

13. Have you read or cited papers from the journal under consideration?

If you are not familiar with work in the journal from your own literature searches, read some articles to assess their quality and relevance.

14. Is the publisher a member of the Committee on Publishing Ethics (COPE) or OASPA (Open Access Scholarly Publishers Association)?

This information may not be easy to find as some smaller, good quality journals may have membership under an umbrella agreement with a larger, co-publisher.

15. Does the publication have an ISSN (International Standard Serial Number)?

Be aware, that this, in itself, is not necessarily an indicator of a quality journal. 

16. Do published articles have permanent identifiers such as Digital Object Identifiers (DOI)?

This should guarantee long-term access online, even if a journal moves from one publisher to another.

17. Are the published articles archived in LOCKSS, CLOCKSS, Portico, a national library or other place that will guarantee perpetuity of access?

A journal’s archiving policy should be stated on its website.

18. Does the journal have a Clarivate Analytics Impact Factor (IF) (previously the Thomson-Reuters Journal Citation Reports (JCR) Impact Factor)?

Not all quality journals have an Impact Factor but if one is stated it must be the industry recognised Clarivate Analytics IF. 

Other checklists

19.Think, Check, Submit.

This is a simple checklist that researchers can use to assess the credentials of a journal or publisher. It is a campaign founded by a number of reputable organisations and can be found at http://thinkchecksubmit.org/

20. If the journal is open access, is the journal listed in DOAJ (Directory of Open Access Journals)?

This directory pre-screens journals prior to their inclusion but should not be relied on exclusively. Journals selected as a potential publishing venue from this directory should also undergo rigorous examination using other criteria.

Key issues in academic publishing

The online environment has transformed scholarly publishing.  Traditionally, communication of the results of research was through the medium of printed publications, paid for by readers via subscription fees, although often subsidised through page charges to authors in order to help keep subscription costs down. 

The rise of Open Access and pay to publish

Journal publishing has undergone rapid change, fuelled, in part, by the ease of digital dissemination of past and current content and by the increasing emphasis on open access (OA) publication. The internet has allowed the emergence of a large number of online-only journals and this move from print to online publishing has changed the costs associated with publishing: from printing and distribution costs to software development, bandwidth, security and archiving costs.

The OA movement calls for the results of publicly funded research to be readable, online, with no subscription fee. The result has been the costs of publication passing from the reader, in the form of subscriptions, to the author of the work, often in the form of article processing charges (APCs). This model, known as the ‘gold’ model of OA, has been the dominant model of Open Access in recent years.

There are other means of providing free access to research. These include pre-print servers and institutional repositories. However, access to research through these alternative sources is not usually to the publisher’s ‘version of record’, but may be to either a non-peer-reviewed version (pre-print) or peer-reviewed and accepted version (post-print). Sometimes, publishers set embargo periods for access to post-prints.

New entrants in the scholarly information environment

Many commercial enterprises have entered the scholarly publishing arena, setting up online journals that accept a fee, in the form of an APC, in return for publication.  However, a number of these do not satisfy the requirements of an academic publishing enterprise. The term ‘predatory publishers’ has been coined for these companies, although the term itself has been criticised. The term ‘predation’ implies intent, whereas some newly emerging publishers may only lack experience in the scholarly publishing industry and the standards required. Other terms suggested for this type of publishing enterprise are ‘dubious’ or ‘low quality’.

Practices of concern include: the masking of fees until after an author’s article is accepted and then refusing the author the option to withdraw their work, or charging a further ‘retraction fee’; not providing expert peer-review of research submitted; or failing to provide the long-term access and archiving of published content that allows work to be discovered, used, and built upon in perpetuity. Some of these publishers erroneously claim that their journals are indexed in scholarly databases, or declare an impact factor that is not the industry-recognised Clarivate Analytics Impact Factor (previously the Thomson-Reuters Journal Citation Reports (JCR) Impact Factor).

Some scholars, believing in the principle of Open Access, have agreed to become editorial board members or peer-reviewers on some of these publications without knowledge of their practices, and have since been unable to have their names removed from the boards. The journals look to be quality publications because of the names associated with them.

Another issue of concern is the taking over of the websites of respected journals by these enterprises: domain names have been bought by untraceable companies and websites set up to look like the original journals, but the journal charges fees and publishes work that the bona fide journal may not have accepted under its normal peer-review process. Another arguably unethical practice may be a claim, by a journal or publisher, of an affiliation to a scholarly society without the knowledge of that organisation.

In addition to setting up large numbers of journals, some new entrants into the scholarly information environment have branched out into running conferences and many scholars have received email invitations to speak at or attend such conferences.

The effect on the scholarly record

One of the more serious consequences of the proliferation of these naïve and potentially unethical publishers is that the scientific literature may become diluted with non-peer-reviewed and, potentially, poor quality content. This may make it difficult for a reader who is not an expert in the field to discriminate between science and pseudo-science. A journal, and hence its content, may look believable to an uninformed readership, perpetuating pseudo-science that is merely opinion and not backed up by evidence. There is a risk of public confidence in the research literature and, potentially, of scholarship in general, being undermined.

A risk that good research becomes irretrievable

A lack of indexing of these journals in databases that provide access to the scholarly literature poses a problem for the research community. If good quality research is published in one of these journals by an unsuspecting researcher it may not be found by others searching in the relevant scholarly databases. There is a risk of research being needlessly duplicated, at a cost to the taxpayer and to the research community. 

A risk that poor quality research is published

Research that is not peer-reviewed may be at risk of being of lower quality than research that benefits from the input of reviewers who are experts in the field of study. The publication of this research in journals run by publishing enterprises that do not adhere to traditional peer-review practices may be criticised for wasting both valuable research funding and the time of research participants.

Note to users

The information in these guidelines has been produced by the Royal Society Te Apārangi to guide researchers when selecting publications.  It has been widely peer-reviewed and thus is likely to portray good practice.  Because it is not formal advice, those who use it need to apply their own judgement to decide what parts of the content are applicable in their own circumstances. 

Acknowledgements

We would like to thank the following reviewers who provided helpful feedback on this document: Lai Kei Pang, Howard Amos, Krassie Petrova, Marcus Williams, Emma Skellern, Susan Eady.

Further information and reading

Addressing Faculty Publishing Concerns with Open Access Journal Quality Indicators. Beaubien, S. & Eckard, M. 2014.  Journal of Librarianship and Scholarly Communication. 2(2), eP1133. DOI: http://doi.org/10.7710/2162-3309.1133 (accessed 3 Nov 2017)

Everything you ever wanted to know about predatory publishing but were afraid to ask. Berger, M. 2017. In At the Helm: Leading Transformation: The Proceedings of the ACRL 2017 Conference, March 22–25, 2017, Baltimore, Maryland, edited by Dawn M. Mueller. Chicago: Association of College and Research Libraries, 2017. http://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1142&context=ny_pubs (accessed 3 Nov 2017)

Who is actually harmed by predatory publishers? Eve, M.P. & Priego, E. 2017. Triple-C Communication, Capitalism & Critique – Journal for a global sustainable information society. 15(2), 755–770 http://www.triple-c.at/index.php/tripleC/article/view/867/1042 (accessed 3 Nov 2017)

Ethical issues in publishing in predatory journals. Ferris, L.E. & Winker, M. A. 2017. Biochem. Med. 27(2) 279–285 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5493173/ (accessed 3 Nov 2017)

Discriminating between legitimate and predatory open access journals: report from the International Federation for Emergency Management Research Committee. Hansoti, B., Langdorf, M.I., Murphy, L.S. 2016.  Western Journal of Emergency Medicine 17(5) 497–507 https://doi.org/10.5811/westjem.2016.7.30328 (accessed 3 Nov 2017)

Stop this waste of people, animals and money. Moher et al, 2017. Nature 549, 23–25 https://doi.org/10.1038/549023a (accessed 3 Nov 2017)

Statement on article publication resulting from NIH funded research. National Institutes of Health. 2017.  NIH Notice files, number NOT-OD-18-011. https://grants.nih.gov/grants/guide/notice-files/NOT-OD-18-011.html (accessed 6 Nov 2017)

‘Predatory’ open access: a longitudinal study of article volumes and market characteristics. Shen, C. & Björk, B. 2015. BMC Medicine 13:230 https://doi.org/10.1186/s12916-015-0469-2 (accessed 3 Nov 2017)