Reviewing for the ISJ

This page is for Referees of ISJ, i.e. those invited by one of our Editors to review a paper. If you wish to become a referee then see here.

If you are a new referee for the Information Systems Journal (ISJ) welcome as a member of our Panel of Referees and thank you for agreeing to review for us. We hope you find this page helpful. If you are an existing referee then probably you will not require this information although it might be of interest. Anyway, we would like to take this opportunity to thank you for your contribution, both now and in the past. We much appreciate our referees and fully recognise the contribution they have made to the success of the ISJ.

A copy of the ISJ Review Evaluation Form is available here.

We provide below some general guidelines for ISJ referees. For those interested in more detail we recommend the paper by Alan Jay Smith* which concerns the tasks of the referee, click here for a copy in pdf format. It is written for the domain of computer science and engineering but is equally relevant for information systems, in our opinion.

If you do not know that much about the ISJ we suggest you first look in detail at other areas of this website. We hope you find it interesting. It is designed to inform authors, reviewers, etc. about the journal and we would welcome any comments or suggestions for improving it.

ISJ operates a pre-review filter for all submissions where unsuitable, inappropriate, or out of scope papers are rejected prior to them being sent out for review. So any paper you are asked to review has already been screened. However, if you feel that despite this a paper reaches you which is not appropriate to send to referees then please contact the Managing Editor or Senior Editor.

ISJ operates a ‘double blind’ reviewing system so you should not know the identiy of the author. However, if you have any reason to think there might be a conflict of interest then you should inform the Associate Editor or Editors. We seek independent and objective reviews. ISJ follows the AIS Code of Research Conduct.

Of particular interest to us, as editors, are for you to address the following questions:

  • is the paper appropriate for the ISJ? The aims and objectives of ISJ are also on this website – see here. Although we draw the boundaries of information systems quite broadly we do not publish purely technical papers nor papers where there are significant other outlets, e.g. pure database papers. See also the information for authors on submitting a paper. The ISJ is primarily a qualitative journal but we do publish some quantitative papers but these must be particularly strong on context and implications rather than just about the quantitative issues. If you think the paper is not appropriate for the ISJ it is useful if you can suggest an alternative outlet.
  • is the paper of interest and importance in the field of information systems? We wish to publish interesting papers rather than arcane pieces of research which have no great relevance. We are amenable to polemical papers as well as the more traditional independently conducted research paper (as long as the first is not purporting to be the second).
  • is the paper well written, well constructed and understandable? Even the best research should be well communicated.
  • is the paper based on an understanding and knowledge of what has previously been published in the area or are there some omissions that are important? This is not to say that a paper must have an extensive literature review of everything in the area. What is necessary is that appropriate references are covered and that the authors have shown what others have found, or said, and if these are supportive or contradictory. As an international journal we are keen to see a wide range of sources used, not just from the country or area of origin of the paper. For example, papers from the US sometimes tend to ignore research in other areas of the world, e.g. Europe or Australasia, and visa versa. We also ask you to check that papers reference existing ISJ publications, where appropriate. It seems ridiculous for us to publish papers on a topic on which there have been contributions in the ISJ which are ignored. We like research to be cumulative. If you need to check what has previously been published in ISJ you can go to the Publishers website and search titles, abstracts, and keywords.
  • is the paper based on a primary research study? We mainly seek to publish primary work. If the research is secondary is it appropriate and solid. We do not usually publsih literature survey papers unless they of particularly quality and contribution.
  • is the research method applied appropriate and justified. Are you confident that it can lead to the findings provided. We believe the research method(s) adopted is important in information systems. Do not reject just because the research method is not your particular favourite, providing it is an appropriate IS method. We are pluralistic in our views in this area so please judge more on its appropriateness, justification and operationalisation.
  • is the paper accurate, as far as one can tell, and are the assumptions on which it is based reasonable? If not we wish to know. Does the research process seem to be adequate and are the conclusions legitimate on the basis of the research undertaken? is the research set in the appropriate context and are the findings made relevant to that context?
  • is the title of the paper appropriate? Is it an adequate description, is it likely to generate interest in the paper, and especially, is it misleading?
  • is the paper within our guidelines with regard to length? We try to keep to a maximum of 7,000 words.
  • is there an abstract and is it an appropriate description of the paper?
  • are there keywords and are they appropriate (good abstracts and keywords are important for the paper to be found by search engines and to be cited)?

If the paper is specifically designated a ‘practitioner’ paper, which we are keen to encourage, we recommend that you do not apply quite the same stringent approach that you would to an academic paper. We are not so concerned with the grounding in theory nor the literature review but more whether it is saying anything new, is it of interest, of relevance, and communicates well.

Do not be afraid to express your views, indeed that is what we wish to hear. There will be at least one other referee of the paper and the views of the Associate Editor as well, so you are not on your own. If your recommendation is to accept with either minor or major changes, your comments should help enable the author(s) to make it an even better paper. It is useful if you make a distinction between those aspects which you think are essential before it could be published and those that would be ‘nice’ but perhaps not absolutely essential. It is usual to provide some general comments and then perhaps some more detailed points. If you suggest literature that you think the authors should address please be as specific as possible in giving the author guidance and references.

If you accept or reject the paper outright then please also make some comments as to why you have done so. Referees sometimes find it difficult to reject outright a paper that they are basically unhappy with but instead they ask for major changes. Such changes are often not really practical, or are tantamount to a different paper. In such circumstances it is usually better to reject.

Please complete the evaluation form and also provide any additional comments you wish to communicate to the editors. Please also provide detailed comments to go to the author(s). Do not identify yourself on this sheet as we operate a double blind refereeing process.

Please return the evaluation form and the comments for author(s) to the Associate Editor (or Guest Editor, if it is an ISJ Special Issue).

We hope these comments help you in your task and thank you again for your efforts on behalf of the ISJ and the information systems community as a whole. We recognise that refereeing papers is a time consuming task and that everyone is very busy, but hopefully sometimes they are also stimulating and challenging. As an attempt to provide you with an element of feedback we send you the comments of both referees, when they are available. This feedback has received much praise from our referees who like to compare their thoughts and assessments with others.


* A. J. Smith (1990) The Task of the Referee, Computer, Vol. 23, No. 4, pp65-71. Available on this site by kind permission of the Editor.