Process and Pitfalls in Writing Information Visualization Research Papers - Information Visualization: Human-Centered Issues and Perspectives - page 136

- algorithm complexity analysis
- implementation performance (speed, memory)
- quantitative metrics
- qualitative discussion of result pictures
- user anecdotes (insights found)
- user community size (adoption)
- informal usability study
- laboratory user study
- field study with target user population
- design justification from task analysis
- visual encoding justification from theoretical principles
In any particular paper, the constraints of researcher time and page limits force
authors to select a subset of these approaches to validation. The taxonomy of
paper types below can provide you with considerable guidance in choosing ap-
propriate validation approaches, leading to a paper structure where your results
back up your claims. The five paper types guide the presentation of your research
by distinguishing between the following possibilities for your primary contribu-
tion: an algorithm, a design, a system, a user study, or a model.
2.2 Technique
Technique papers focus on novel algorithms and an implementation is expected.
The most straightforward case is where the research contribution is a new algo-
rithm that refines or improves a technique proposed in previous work. A typical
claim is that the new algorithm is faster, more scalable, or provides better visual
quality than the previously proposed one. The MillionVis system [5], hierarchical
parallel coordinates [6], and hierarchical edge bundling [15] are good exemplars
for this category.
Typical results to back up such a claim would be algorithm complexity anal-
ysis, quantitative timing measurements of the implementation, and a qualitative
discussion of images created by the new algorithm. Quantitative metrics of image
quality, for example edge crossings in graph layout, are also appropriate. You
need to compare these results side by side against those from competing algo-
rithms. You might collect this information through some combination of using
results from previous publications, running publicly available code, or imple-
menting them yourself. In this case, there is very little or no design justification
for whether the technique is actually suitable for the proposed problem domain
in the paper itself: there is an implicit assumption that the previous cited work
makes such arguments.
In retrospect, a better name for this category might be Algorithms .Many
authors who design new visual representations might think that a paper docu-
menting a new technique belongs in the Technique category. However, the ques-
tion to ask is whether your primary contribution is the algorithm itself, or the
design. If your algorithm is sophisticated enough that it requires several pages
of description for replicability, then you probably have a primary algorithmic