Paper reviews instructions

538B Distributed Systems: Project proposal instructions

Winter 2019

Your response should (1) reflect your understanding and analysis of the issues raised by the paper, and (2) should help direct your and others' preparation for in-class discussion.

You may write the response in whatever style you prefer that meets the above goals. One good format for the response is to critique the paper, listing the following three points: its biggest contribution (and, briefly, why that result was not already obvious), its biggest mistake (in motivation, methodology, algorithm, data analysis, conclusions, or some other area), and the biggest question that it raises (or the most interesting and important follow-on work that it suggests). Another acceptable format is to summarize the paper, describing its thesis, approach, and conclusions, and stating why it is significant. The commentary should also list questions that you have about the paper, such as about technical points or connections to related work.

It's OK if you read the paper and there are issues that you do not understand. Please ask questions about those issues -- both in your response and in class -- and we will all gain by the discussion. It is best to explain why something makes no sense to you. For example, don't just say, "I didn't understand section 2", but state where there is a logical fallacy or a conclusion that does not follow or a term that is not defined. Your questions will help shape the lecture.

If you have a question, it is likely that many other people have the same question; they will appreciate your raising the point. However, do come to class prepared: carefully read the paper and get as much as you can out of it on your own. Doing so will make the class time much more productive.

Write your response with an eye towards contributing something interesting to the discussion in class the following day.

List of questions to help you write you response.
  • Why was the paper written? What problem does it solve? Does it completely solve the problem; under what circumstances? Why is the problem interesting/important?
  • How is the paper topic relevant to us today?
  • What question would you want to ask the authors, and why?
  • Why are we reading this paper? What can we take away from the paper? How did the paper change your perspective on distributed systems?
  • Is the paper the final word on the subject? What are some questions that it brings up?

Other bits to keep in mind.

  • I am interested in reading about what you think, not what the authors think. Therefore, a successful response would primarily revolve around your ideas/thoughts about the paper.
  • Make your words count. A longer response is not necessarily a better response.
  • Focus on a few specific, technical, aspects of the paper. Avoid making general claims/comments about the paper.
  • Spend the time to thoroughly discuss the issues/ideas that you do raise. It is better to have 1 interesting well-discussed idea, rather than 5 short and fragmented bullets. Aim for a cohesive narrative.
  • Avoid meta comments (e.g., "paper needs more figures"). If you want to make a meta comment/question about the reading, then make it as specific as possible and interesting to the rest of us.
  • When critiquing an aspect of the paper (e.g., no message loss/FIFO queues/etc are unrealistic) consider why the authors ignored these issues (perhaps these are not fundamental to the problem that they are solving?). It also helps to think about how you would overcome these issues (again, I'm much more interested in your thoughts/solutions, then blanket criticisms).
  • Every paper has shortcomings, so if you do bring up a shortcoming, then make sure that your response discusses/reflects on the shortcoming in a non-trivial manner. For example, simply stating that "it was not clear how 30s delay was computed" is not sufficient. You need to talk about why you think the authors decided not to solve this problem, why this problem is worth solving, what are the implications of not solving this problem in this specific system, etc. Finally, when you do bring up shortcomings, make sure to bring up shortcomings that are fundamental, and not those that can be easily remedied.