Guidelines for the evaluation of SoCG submissions
Relevant topics
See the call for papers for a non-exhaustive list of topics.
Evaluation criteria — paper types
When evaluating a SoCG paper, it is important to keep in mind that there are different types of contributions, each with their own strengths. To ensure that each paper is evaluated on its own merits, identify the main strengths of the submission, as captured by four possible paper types. These paper types are described in detail here, together with their associated evaluation criteria. There are no quotas for the paper types and submissions can be labeled with more than one paper type.
Score ratings and their meaning
Score | Interpretation |
Strong accept (+3) | An enthusiastic accept. An excellent paper — advances the field in an important way — well written and makes it easy to understand the significance of the results. People should definitely attend the talk. This should be among the top third of the accepted papers. I would fight strongly for this paper. |
Accept (+2) | A solid contribution. I feel I learned something worthwhile from this paper. I would want to go to the talk. Should be in the middle third of accepted papers. |
Weak accept (+1) | A weak vote for acceptance. A reasonable contribution to an interesting problem — or maybe the contribution is good but the authors don't express it well — or maybe it's a good paper, but the subject area is marginal for the conference. Not a stellar result, but worth accepting. Should be in the bottom third of accepted papers. |
Borderline (0) | Ambivalent. Probably publishable as a journal paper in a medium journal, but a bit too specialized or too incremental for SoCG. Or perhaps it has nice ideas but is too preliminary, or too poorly written. Please try to refrain from giving this score. |
Weak reject (-1) | A competent paper, but not of sufficient interest/depth for SoCG. A weak to moderate vote for rejection, but I concede that other people see some merits in the paper. |
Reject (-2) | Too preliminary / badly-written / making-such-a-minor-improvement-on-such-an-esoteric-topic. I would fight to have this paper rejected from the conference. |
Strong reject (-3) | A poor paper, unsuitable for any journal. Trivial and/or non-novel and/or incorrect and/or out of scope. |
Confidence ratings and their meaning
Grade | Interpretation |
Expert (5) | Expert. Consider me an "expert" on this paper. I understand it in detail. I know the field, and I am perfectly sure about my judgement; I have checked and understood all proofs. |
High (4) | High. I am fairly familiar with the area of this paper, and have read the paper closely enough to be reasonably confident of my judgment. |
Medium (3) | Medium. I have read the paper carefully and understood the main ideas, but I'm not very confident of my judgment on it. |
Low (2) | Low. I am not an expert. My evaluation is that of an informed outsider. I have some idea of what this paper is about, but I'm not all that confident of my judgment on it. |
None (1) | None. (to be avoided...!) Please do not use this except in extreme circumstances. |
Appendix
Every submission must provide all details needed to verify the results. Supporting materials, including proofs of theoretical claims and experimental details, that do not fit in the 12-page limit should be given in an appendix. If more appropriate, the full version may be given as the appendix. In both cases, however, the authors should include in the main part specific pointers to the relevant locations in the appendix. A paper without the appendix should be able to stand on its own.
Ethical Issues
Submitted papers are confidential. We must not distribute them, or use them for our research. Similarly, your reviews, grades, and confidence scores must be kept confidential. Submissions should be judged solely on the basis of the submitted extended abstracts. You may have a personal bias on some papers. The reasons are many—personal/professional ties to authors, you or your student are just working on the same problem, etc. Only you can judge such a bias, and decide if you don't feel comfortable reviewing the paper. In this case, mention a conflict of interest to the PC.