538B Distributed Systems: Paper reviews advice
Reviews should engage with the reading at a deep level. Write you
review with an eye towards contributing something interesting to the
discussion in class the following day.
List of questions to help you write you review.
- Why was the paper written? What problem does it solve? Does it
completely solve the problem; under what circumstances? Why is the
- How is the paper topic relevant to us today?
- What question would you want to ask the authors, and why?
- Why are we reading this paper? What can we take away from the
paper? How did the paper change your perspective on distributed
- Is the paper the final word on the subject? What are some
questions that it brings up?
- Your review mark depends primarily on the content in the detailed
review field, not the summary field.
- The summary of the paper should go into the summary field. The
review should go into the review field. Avoid reviews that are
summaries of the paper. In your review I want to see that you thought
critically about the paper and what the paper has to offer.
- I am interested in reading about what you think, not what the
authors think. Therefore, a successful review would primarily revolve
around your ideas/thoughts about the paper.
- Make your words count. A longer review is not necessarily a better
- Focus on a few specific, technical, aspects of the paper. Avoid
making general claims/comments about the paper.
- Spend the time to thoroughly discuss the issues/ideas that you do
raise. It is better to have 1 interesting well-discussed idea, rather
than 5 short and fragmented bullets. Aim for a cohesive
- Avoid meta comments (e.g., "paper needs more figures"). If you
want to make a meta comment/question about the reading, then make it
as specific as possible and interesting to the rest of us.
- When critiquing an aspect of the paper (e.g., no message loss/FIFO
queues/etc are unrealistic) consider why the authors ignored these
issues (perhaps these are not fundamental to the problem that they are
solving?). It also helps to think about how you would overcome these
issues (again, I'm much more interested in your thoughts/solutions,
then blanket criticisms).
- Every paper has shortcomings, so if you do bring up a shortcoming,
then make sure that your detailed review discusses/reflects on the
shortcoming in a non-trivial manner. For example, simply stating that
"it was not clear how 30s delay was computed" is not sufficient. You
need to talk about why you think the authors decided not to solve this
problem, why this problem is worth solving, what are the implications
of not solving this problem in this specific system, etc. Finally,
when you do bring up shortcomings, make sure to bring up shortcomings
that are fundamental, and not those that can be easily remedied.