Guidelines for Reviewers - full papers

Introduction

Full papers submitted to the ATN Assessment Conference 2011 underwent a rigorous double blind peer review.

Reviews took into consideration the overall theme of this conference, Meeting the challenges, and the four strands relating to this central theme:

  1. Assessment and standards
  2. Leadership and assessment
  3. Practical solutions to challenging problems
  4. Student engagement in assessment

Appointment and acknowledgement of reviewers

Reviewers were appointed by invitation on the basis of their expertise and experience in areas relevant to the conference. All reviewers were acknowledged in the published proceedings and online in the conference website.

The role of the reviewer

The role of the ATN Assessment reviewer was threefold:

In brief, the DEEWR standards for research include pure basic research, strategic basic research, applied research and experimental development. They require that the paper be original and have the potential to produce results that are sufficiently general for theoretical and/or practical knowledge to be recognisably increased. Most higher education research work would qualify as research.

Double blind reviewing

The double blind review process met DEEWR requirements for peer review of research publications. Reviewers received a paper devoid of the authors' names and institutions in order to ensure objectivity and anonymity.

Reviewers who thought their objectivity was compromised by inadvertently identifying an author, were required to email the conference committee immediately (atna2011@curtin.edu.au) so the paper could be reassigned.

The review process

All reviewers were required to respect the confidentiality of the process.
In reviewing the papers reviewers were required to rate them on the basis of:

Reviewers judged the submission using these criteria and the evaluation guide provided below. There was also provision for reviewers to provide constructive formative feedback to authors so that authors could improve their submission. Reviewers were also able to provide confidential comments for the conference committee, if required.

Papers rejected for a refereed publication were able to be considered for inclusion in the conference as a non-refereed contribution. Reviewers were able to recommend to the conference committee that a full paper may be more appropriately presented as a short paper (abstract only) or poster.

The conference committee assessed the two reviews for each paper to determine the outcome. If reviews were substantially different a third review was called for. Once this process was complete the committee advised the author of the decision, and released aggregated reviewer comments to the author.

Reviewers who had reason to suspect that a contribution was plagiarised, were required to contact the conference committee immediately.

Evaluation Guidelines for Reviewers

Click here to open the guidelines for reviewers.

Making recommendations for publication and giving feedback

The following is the guide used for making recommendations for acceptance into the conference and subsequent publication in the conference proceedings.

  1. All criteria are evaluated as 'Excellent' or 'Acceptable'

The final outcome should be 'Accept as is'.

Do not worry about occasional minor spelling, grammatical or formatting errors. All accepted papers will be scrutinised by the editorial team to ensure that their final submitted paper adheres to formatting requirements prior to publication.

Feedback: Papers accepted as is should be given feedback on the positive qualities of the paper. Please advise them if they will need to attend to spelling, grammar or formatting.

  1. Some criteria are evaluated as 'Borderline' (but none as 'Unsatisfactory')

The final outcome should be 'Accept with revisions', but you may also choose to reject and recommend acceptance in a different (non-refereed) category (see below).

Accept the paper with revisions where you feel less than 20% of the paper needs re-working and/or where the paper needs to be reduced to fit the limit on paper length.

Feedback: Papers accepted with revisions should be given feedback on the positive qualities including a clear identification of the areas where improvement is required. Also please let authors know if, in addition, they will need to attend to spelling, grammar or formatting.

  1. Paper is evaluated with one or more criteria judged to be 'Unsatisfactory'

The final outcome should be 'Reject', but you may also choose to reject and recommend acceptance in a different (non-refereed) category (see below).

Reject the paper or proposal where you feel that it does match the required standard for a peer reviewed paper in this conference.

Rejecting a paper and recommending acceptance in a different (non-refereed) category

Where a paper does not warrant publication as a refereed article, you may wish to reject the paper and 'Accept as short paper (abstract only)' or 'Accept as poster' if you feel that the audience would benefit from the presentation.

Authors should be given feedback on the positive qualities and suggested areas for improvement. Ensure that you clearly indicate the reason why their paper was not suitable as a refereed contribution.

Back to top