538B Distributed Systems: Paper reviews advice

Fall 2016

Reviews should engage with the reading at a deep level. Write you review with an eye towards contributing something interesting to the discussion in class the following day.

• Why was the paper written? What problem does it solve? Does it completely solve the problem; under what circumstances? Why is the problem interesting/important?
• How is the paper topic relevant to us today?
• What question would you want to ask the authors, and why?
• Why are we reading this paper? What can we take away from the paper? How did the paper change your perspective on distributed systems?
• Is the paper the final word on the subject? What are some questions that it brings up?

Helpful advice.

• Your review mark depends primarily on the content in the detailed review field, not the summary field.
• The summary of the paper should go into the summary field. The review should go into the review field. Avoid reviews that are summaries of the paper. In your review I want to see that you thought critically about the paper and what the paper has to offer.
• I am interested in reading about what you think, not what the authors think. Therefore, a successful review would primarily revolve around your ideas/thoughts about the paper.
• Make your words count. A longer review is not necessarily a better review.
• Focus on a few specific, technical, aspects of the paper. Avoid making general claims/comments about the paper.
• Spend the time to thoroughly discuss the issues/ideas that you do raise. It is better to have 1 interesting well-discussed idea, rather than 5 short and fragmented bullets. Aim for a cohesive narrative.
• Avoid meta comments (e.g., "paper needs more figures"). If you want to make a meta comment/question about the reading, then make it as specific as possible and interesting to the rest of us.
• When critiquing an aspect of the paper (e.g., no message loss/FIFO queues/etc are unrealistic) consider why the authors ignored these issues (perhaps these are not fundamental to the problem that they are solving?). It also helps to think about how you would overcome these issues (again, I'm much more interested in your thoughts/solutions, then blanket criticisms).
• Every paper has shortcomings, so if you do bring up a shortcoming, then make sure that your detailed review discusses/reflects on the shortcoming in a non-trivial manner. For example, simply stating that "it was not clear how 30s delay was computed" is not sufficient. You need to talk about why you think the authors decided not to solve this problem, why this problem is worth solving, what are the implications of not solving this problem in this specific system, etc. Finally, when you do bring up shortcomings, make sure to bring up shortcomings that are fundamental, and not those that can be easily remedied.