Hyang-Ah Kim (hakim@cs.cmu.edu) --------------------------------------------- How should system papers be written? (After reading "An Evaluation of Ninth SOSP Submission") System papers (especially those submitted to SOSP or similar conferences) are fallen into one of the following three categories: an introduction to a newly developed real system, a new idea unimplemented yet but worth being considered in future system development, and a topic in theoretical areas. "An Evaluation of Ninth SOSP Submission" listed some valuable tips for writing system papers and the properties the system paper should have. The topic itself should be original and the authors should do their best to prove the originality. Good system papers present the implementation or at least the possibility of realization. The papers should show the alternatives and why they made such a decision. Also, good papers should be able to give lessons to readers. The clear and logical presentation must be a general rule for any kind of paper. Here, I want to add some more to those lists. 1) Clear design principles What I often complain, when I read system papers, is many authors are too busy just describing the resulting systems. They don't mention the design principles they used to make a choice among many alternatives. They even don't show what alternatives they've thought of. There are many ways to build a system and the difference in the rationale, design principles, or the primary goal of the system results in different systems. Therefore, I think, the system papers should make the design principles clear before diving into the detailed description on the systems. Moreover, the sufficient description on design alternatives gives readers more impression that the authors thoroughly studied the problem. 2) Introduction The introduction is most important part of papers. Most readers will not go over the introduction if the introduction does not motivate the readers enough and precisely describe the problem to be addressed. However, many papers just waste the first one page describing earlier works or history. I often see many papers starting with similar sentences, and get bored immediately. The realistic motivation, the precise and concise problem/idea description, and promising results need to appear as early as possible. Explicitly mentioning the key points in the introduction helps the reader to understand the main body of the paper, too. 3) Realistic assumption and workload The importance of realistic workload cannot be more emphasized. Motivation should come from the careful observation on the real world and so do the assumptions the system is based on. Authors should try to show the real world could benefit from their systems. Workload has to be as realistic as possible. When performing simulations with synthetic workloads,- yes, sometimes they are useful to emphasize the system's strength - the workloads should be represent major characteristics of the real workload. Optimization based on the real workload analysis is very persuasive. The authors of AFS paper did a good job implementing and optimizing based on the real usage of the system. 4) Importance of related work and reference list The reference list exists not only to point the source of the referred sentence. It serves as the indicator about how much the authors understand the subject and how fully they studied and considered. Related work is also important to contrast the work against other previous works. Do not just list the name. Do point out what the difference is, what their goals were, and why my work is new. I liked the related work chapter of ABACUS paper and NASD paper since they clearly pointed out the differences from previous works.