User Services Office

PEER REVIEW MEMBER’S GUIDE

Below please find details about the review process and evaluation criteria for rating proposals.

Timeframe –

Conflict of Interest – CLSI requests all reviewers to practice ethical behavior and to disclose real, potential, or apparent conflicts of interest. A reviewer is in conflict if he/she could personally benefit from a decision or when private interests conflict with public duties. If you are in conflict of interest or not available to perform the review prior to the deadline, please “Decline Peer Review”.

Evaluation Criteria and Rating Scales

Each proposal is reviewed and scored by at least three external reviewers and at least one member of the Peer Review Committee.  All reviewers have an opportunity to request clarifications from the applicant if required during the review process, using the electronic proposal system.  Reviewers are asked to provide an integer score in each of the three Evaluation Criteria, as described below:

Quality of scientific research in the context of the field
Does the proposal describe what is to be studied and the importance of it? What hypothesis would be tested, how will the results impact the field, and what is the likelihood of success?

  • 1 – Exceptional:  The proposal describes highly innovative research of great importance. The proposed research will significantly advance knowledge in a specific field or scientific discipline and may have considerable societal relevance.  This is a model for a well written proposal.  High-risk experiments with exceptional potential can be rated in this category at the discretion of the reviewer.
  • 2 – Excellent:  The proposal describes research that is of very high quality and will likely make a significant contribution to a specific field or scientific discipline. As cutting edge research it has the potential to be published in a top-tier scientific journal. 
  • 3 – Good:  The proposal describes research that is likely to produce publishable results in specialized but not top-tier journals. 
  • 4 – Fair:  The proposal describes research that is of limited scope and will not significantly impact a specific field or scientific discipline. Publication may or may not result from this research. 
  • 5 – Poor:  The proposed research is not well planned or is not feasible, or the proposal is so poorly written it is impossible to judge. The work as described is not likely to result in publication.

Suitability of CLS resources being allocated relative to the proposed research
Is this a good use of CLS resources?  Does the experiment require the resources being requested?  Reviewers are also asked to comment on the appropriateness of the number of shifts that have been requested.

  • 1 – Exceptional:  There is no other way to answer the scientific question posed without using the CLS beamline requested.  The experiment represents optimal use of the beamline.
  • 2 – Excellent:  Synchrotron radiation from a third generation source is required for this project, and the CLS beamline requested is well suited for this type of research.
  • 3 – Good:  Synchrotron radiation is required to accomplish the intended goals of the research, but either another CLS beamline is a better match, the experiment is not taking advantage of the capabilities of a third generation source or the experiment is not well suited for the capabilities of the beamline at this time.
  • 4 – Fair:  The proposal has not adequately justified that a synchrotron experiment is required to answer the scientific question.  In some cases the proposal may be improved by first performing experiments using more conventional techniques.
  • 5 – Poor:  There are non-synchrotron methods which should be used to answer this scientific question.  This is not appropriate use of CLS resources.

Quality and capability of the researchers based on their track record
Does the research team have recent synchrotron and/or other relevant experience? If they are a past user they should have clearly demonstrated their track record and productivity in the proposal.  A list of publications reported to CLS can be viewed here.

  • 1 – Exceptional:  The applicants have demonstrated a highly productive track record of publishing their CLS results in top-tier journals.  In the case of researchers new to the CLS, they show exceptional potential.
  • 2 – Excellent:  The applicants have consistently published their CLS results, sometimes in top-tier journals. In the case of researchers new to the CLS, there is a very high likelihood that these researchers will become highly productive users.
  • 3 – Good:  The applicants have a reasonable record of publishing their CLS results but rarely in top-tier journals, or they are potentially productive past users who have demonstrated that they should be given more time.  In the case of researchers new to the CLS, they have demonstrated potential for becoming productive users.
  • 4 – Fair:  The researchers have only been marginally productive with previous CLS beamtime and there is need for improvement. In the case of researchers new to the CLS, their proposal reveals a lack of knowledge of synchrotron experiments, and they could benefit from more training and/or mentoring.
  • 5 – Poor:  Researchers have received significant beamtime at CLS with an unacceptable publication record.

To aid reviewers further, the rating scale can be calibrated as follows:

Results of the last review are available here.

Beamline Characteristics – 01B1-1 Mid-IR, 02B1-1 Far IR, 06B1-1 SXRMB, 06ID-1 HXMA, 07B2-1 VESPERS, 08ID-1 CMCF1, 10ID-1 SM, 11ID-1 SGM, 11ID-2 PGM.  View summary or current status

Process- The review process has been established using an electronic workflow system to submit and review proposals and provide applicants with the results. The workflow generates an email notification informing the reviewer that a workflow item needs their attention and to logon to https://user.lightsource.ca

The electronic review provides:

It is acceptable to:

Once your review is submitted your rating and comments cannot be modified.

If you encounter any difficulties or have any questions, please contact our office clsuo@lightsource.ca or 306-657-3700.

Last modified: 2012-01-19 17:01:32