User Services Office

About the Peer Review

User time obtained through General User Access is via competitive peer review two times per year emphasizing the excellence of science. (click here for details)

Evaluation Criteria
Each proposal is reviewed and scored by at least three external reviewers and at least one member of the Peer Review Committee for “All Other Beamlines” and at least two committee members for “Macromolecular Crystallography”.  All reviewers have an opportunity to request clarifications from the applicant if required during the review process, using the electronic proposal system.  Reviewers are asked to provide an integer score in each of the three Evaluation Criteria, as described below:

Quality of scientific research in the context of the field
Does the proposal describe what is to be studied and the importance of it? What hypothesis would be tested, how will the results impact the field, and what is the likelihood of success?

  • 1 – Exceptional:  The proposal describes highly innovative research of great importance. The proposed research will significantly advance knowledge in a specific field or scientific discipline and may have considerable societal relevance.  This is a model for a well written proposal.  High-risk experiments with exceptional potential can be rated in this category at the discretion of the reviewer.
  • 2 – Excellent:  The proposal describes research that is of very high quality and will likely make a significant contribution to a specific field or scientific discipline. As cutting edge research it has the potential to be published in a top-tier scientific journal. 
  • 3 – Good:  The proposal describes research that is likely to produce publishable results in specialized but not top-tier journals. 
  • 4 – Fair:  The proposal describes research that is of limited scope and will not significantly impact a specific field or scientific discipline. Publication may or may not result from this research. 
  • 5 – Poor:  The proposed research is not well planned or is not feasible, or the proposal is so poorly written it is impossible to judge. The work as described is not likely to result in publication.

Suitability of CLS resources being allocated relative to the proposed research
Is this a good use of CLS resources?  Does the experiment require the resources being requested?  Reviewers are also asked to comment on the appropriateness of the number of shifts that have been requested.

  • 1 – Exceptional:  There is no other way to answer the scientific question posed without using the CLS beamline requested.  The experiment represents optimal use of the beamline.
  • 2 – Excellent:  Synchrotron radiation from a third generation source is required for this project, and the CLS beamline requested is well suited for this type of research.
  • 3 – Good:  Synchrotron radiation is required to accomplish the intended goals of the research, but either another CLS beamline is a better match, the experiment is not taking advantage of the capabilities of a third generation source or the experiment is not well suited for the capabilities of the beamline at this time.
  • 4 – Fair:  The proposal has not adequately justified that a synchrotron experiment is required to answer the scientific question.  In some cases the proposal may be improved by first performing experiments using more conventional techniques.
  • 5 – Poor:  There are non-synchrotron methods which should be used to answer this scientific question.  This is not appropriate use of CLS resources.

Quality and capability of the researchers based on their track record
Does the research team have recent synchrotron and/or other relevant experience? If they are a past user they should have clearly demonstrated their track record and productivity in the proposal.  A list of publications reported to CLS can be viewed here.

  • 1 – Exceptional:  The applicants have demonstrated a highly productive track record of publishing their CLS results in top-tier journals.  In the case of researchers new to the CLS, they show exceptional potential.
  • 2 – Excellent:  The applicants have consistently published their CLS results, sometimes in top-tier journals. In the case of researchers new to the CLS, there is a very high likelihood that these researchers will become highly productive users.
  • 3 – Good:  The applicants have a reasonable record of publishing their CLS results but rarely in top-tier journals, or they are potentially productive past users who have demonstrated that they should be given more time.  In the case of researchers new to the CLS, they have demonstrated potential for becoming productive users.
  • 4 – Fair:  The researchers have only been marginally productive with previous CLS beamtime and there is need for improvement. In the case of researchers new to the CLS, their proposal reveals a lack of knowledge of synchrotron experiments, and they could benefit from more training and/or mentoring.
  • 5 – Poor:  Researchers have received significant beamtime at CLS with an unacceptable publication record.

To aid reviewers further, the rating scale can be calibrated as follows:

Peer Review Committee

Peer Review Committee (PRC) - Macromolecular Crystallography

Peer Review Committee (PRC) – All Other Beamlines

The members of the PRC may include national and international representation knowledgeable in synchrotron-based science that perform research, manage research or use research. The CLSI Director of Research appoints the Chair and members with recommendations from CLS external advisory bodies (Scientific Advisory Committee, Beamline Advisory Committee and Users’ Advisory Committee) and suggestions from the community at large.
The PRC members:

Review and Allocation Process

Once all reviews have been completed, the Peer Review Committee meets to discuss the results.  Particular attention is paid to proposals showing a high standard deviation.  Once consensus has been reached, proposals will be ranked in order based on scores for the quality of science and suitability.  Proposals with equal scores will be grouped together (binned).  These scores and recommendations with respect to shifts are then passed on to the Allocation Committee, chaired by the CLS Director of Research, and composed of Beamline Scientists and the Assistant Director of Research.  They are responsible for the final allocation of beamtime.  Proposals close to the cut-off score will be closely analyzed, and score for the quality and capability of the researcher may be the deciding factor.

Last modified: 2012-01-19 17:01:18