Thoughts on User Studies: Why, How, and When

URL of this page: http://www.VRVis.at/vis/research/vvp-userstudies/index.html


Thoughts on User Studies: Why, How, and When


Abstract:
Visualization as currently practiced is mostly a craft. Methods are often designed and evaluated by presenting results informally to potential users. No matter how efficient a visualization technique may be, or how well motivated from theory, if it does not convey information effectively, it is of little use. User studies offer a scientifically sound method to measure a visualization s performance. Although their use has become more widespread, we believe they have the potential for a much broader impact. This article describes our experiences with user studies. We offer some examples of our own studies, talk about the pitfalls and problems we encountered, and show how the results were applied to produce successful visualizations. Although our main goal is to encourage the use of studies in visualization, we recognize that other disciplines also offer important insights into visualization design, for example, the areas of visual design or the visual arts. We conclude by discussing when knowledge from these areas might be preferable to a traditional user study.

Project:
This paper was written in close cooperation between

Robert Kosara, BR2, VRVis Research Center, Vienna, Austria
Christopher G. Healey, North Carolina State University (NCSU), Raleigh, NC
Victoria Interrante, University of Minnesota
David H. Laidlaw, Brown University, Providence, RI
Colin Ware, University of New Hampshire, Durham, NH

Paper: An early version of the article is available as VRVis technical report TR-VRVis-2002-045 (8 pages) from January 2003. A much improved version of this report has been accepted for publication in IEEE Computer Graphics & Applications (CG&A), Vol. 23, No. 4, July/August 2003.

Outline:

Introduction
Why Conduct User Studies?

Color Sequences
Shape from Texture
Perceptual Textures
Usability Testing and SDOF
Automated Mini-Studies

When Do User Studies Help?

Beyond User Studies?
When Things Go Wrong

Conclusions
How To Do User Studies (Sidebar)

This page is maintained by Robert Kosara.
In case of questions, comments, etc., please mailto:Kosara@VRVis.at.

Abstract:	Visualization as currently practiced is mostly a craft. Methods are often designed and evaluated by presenting results informally to potential users. No matter how efficient a visualization technique may be, or how well motivated from theory, if it does not convey information effectively, it is of little use. User studies offer a scientifically sound method to measure a visualization s performance. Although their use has become more widespread, we believe they have the potential for a much broader impact. This article describes our experiences with user studies. We offer some examples of our own studies, talk about the pitfalls and problems we encountered, and show how the results were applied to produce successful visualizations. Although our main goal is to encourage the use of studies in visualization, we recognize that other disciplines also offer important insights into visualization design, for example, the areas of visual design or the visual arts. We conclude by discussing when knowledge from these areas might be preferable to a traditional user study.
Project:	This paper was written in close cooperation between Robert Kosara, BR2, VRVis Research Center, Vienna, Austria Christopher G. Healey, North Carolina State University (NCSU), Raleigh, NC Victoria Interrante, University of Minnesota David H. Laidlaw, Brown University, Providence, RI Colin Ware, University of New Hampshire, Durham, NH
Paper:	An early version of the article is available as VRVis technical report TR-VRVis-2002-045 (8 pages) from January 2003. A much improved version of this report has been accepted for publication in IEEE Computer Graphics & Applications (CG&A), Vol. 23, No. 4, July/August 2003.
Outline:	Introduction Why Conduct User Studies? Color Sequences Shape from Texture Perceptual Textures Usability Testing and SDOF Automated Mini-Studies When Do User Studies Help? Beyond User Studies? When Things Go Wrong Conclusions How To Do User Studies (Sidebar)
This page is maintained by Robert Kosara. In case of questions, comments, etc., please mailto:Kosara@VRVis.at.