Authors:
Bill Gaver, Kristina Höök
Fresh out of this year's CHI paper committee meeting, we thought it might be helpful to share a few reflections from the Design subcommittee on our findings. We had time for a group discussion in the hotel meeting room we had shared over a long two days, discussing and debating which papers to accept from well over 100 submissions. After all the dissecting and questioning, both in the room and during breakfasts, lunches, dinners, and breaks, we were all well prepared to think about how CHI design papers can succeed, and where they tend to fall short. Our hope in sharing our observations is to encourage and reassure designers wanting to submit to CHI, to suggest to reviewers what they should look for in submissions, and, in the end, to continue to improve the quality of design papers at the conference.
Rest assured, the quality of this year's papers is at least as good as in years past. Nonetheless, the subcommittee couldn't help but feel a little dissatisfied, even worried. Why do so many submissions seem only tangentially relevant for design? Why are so few "real design papers" submitted to CHI?
Where'S The Design?
We worry that Design is becoming something of a catch-all category at CHI. In many ways, it's a good thing we provide a home for submissions that don't fit easily into other categories, particularly papers that are risky, transdisciplinary, or unconventional. But all too often we encountered submissions that seemed better suited to one of the other subcommittees at CHI: Understanding Users, for example, or Interaction Techniques, Devices and Modalities. Why do these get sent to Design?
Of course, design has always been a difficult discipline to define. After all, we talk about interaction design, but also user interface design, or the design of computer architectures, or the design of data structures. For that matter, it is not unusual to hear that everybody is a designer—though we tend to agree with Bill Buxton's rejoinder that if everybody who chooses their own clothes is a designer, then anybody who can count change is a mathematician.
Scoping design so broadly doesn't seem useful to us. Instead, we think of design as involving certain skills and practices, including, for instance, planning and making finished artifacts, creative processes for reframing problems and developing design spaces, engagement with settings, material explorations, and an attention to aesthetics that seeks not just to make things beautiful but also to convey cultural identity, guide expectations, and shape a dynamic gestalt. Design in this sense may be pursued by individuals, teams, or collaborative groups; design work is increasingly distributed, outsourced, or left open for completion by end users. Nonetheless there is a family resemblance to design practices that Nigel Cross characterized as a way of thinking, involving synthetic, proactive approaches to understanding and shaping the world through artifacts. These are the sorts of practices developed through specialist courses and educational institutions, and though they can be pursued independently, it is important that they speak to those communities: Not everybody who makes something is a designer, and not all studies with relevance to design are design research.
So one reason that authors submit work that doesn't really fit the Design subcommittee may be that they have an overly inclusive understanding of what we mean by design. It may also be, however, that submissions get sent to Design because authors expect an easier ride with us than with other subcommittees. This is not necessarily a question of submitting weak submissions to a subcommittee perceived to be lax. There are rumors that some CHI subcommittees may have become narrow and doctrinal, losing a broad view of their subject and becoming unwilling or unable to accept papers that do not conform to their favored approaches (note that our characterization of design above is not intended as a doctrine!). It is also the case that we evaluate each submission on its merits, instead of penalizing those that are not "Design" for being sent to the wrong subcommittee. This may change, however, at least in the sense that we may start more assertively referring papers that don't belong to Design to other subcommittees. In any case, we suggest that it would be far more effective to change subcommittee cultures from within, rather than sending inappropriate papers to Design.
The Myth of the Perfect CHI Design Paper
The more important question to us, however, is why we don't see more "real design papers." Perhaps one of the reasons for this is that what we on the subcommittee look for in successful design papers seems to differ from what authors expect.
In our discussion, it became apparent that what we mean by real design papers is ones that focus on one or several of the practices listed earlier—the making of artifacts, creative processes, material explorations, or aesthetic crafting. Insofar as such concerns are the heart of design, it makes sense that they should be the topic of design papers as well.
Over the years, however, it seems the community's expectations of CHI design papers have come to include a lot more besides. These days, beyond making, processes, engagements, explorations, and crafting, other ingredients now seem essential. Most people "know," for instance, that papers describing new designs will not be accepted unless they include some sort of user study. Beyond this, CHI design papers should—according to folk wisdom—be framed in terms of an overarching design approach (ideally new and with a catchy name), motivated by a set of specific research questions, accompanied by an extensive literature review, and analyzed in a lengthy discussion to produce generalizable lessons, ideally in theoretical terms.
Trying to write the mythical perfect CHI design paper is daunting. After all, engaging with contexts, developing a design space, exploring materials, and developing a finished artifact—the basic elements of design research—already takes a huge amount of work. If authors believe that they must add to this a formal user test, an articulated research program, a set of research questions, an extensive literature review, and a lengthy discussion of generalizability and theoretical import, then, well, why bother? Why not submit to a more sympathetic venue or exhibition? And if they do bother, and develop all these other aspects of an "ideal" design research paper, then is it any wonder that quality suffers?
All too often one of us would report about a submission: "The design is amazing, but they include a user study that is so flawed we really can't accept the paper." Worse still were submissions that developed convincing programs, great literature reviews, and solid user studies—but with designs that were weak, poorly reported, or entirely absent. Overall, what we saw in the meeting is that frequently, when authors do try to fulfill the mythical requirements of CHI design papers, they don't do a very good job.
Equally dismaying was the fact that, in our discussions, it became clear that none of us actually demand that CHI design papers embody the myth. Think about that: In a room full of expert design researchers, senior in the field, the very people who choose and instruct reviewers and ultimately decide which submissions will be accepted or not, not one supported the idea that acceptable CHI design papers have to contain all the elements of the so-called perfect design paper!
Clearly we have a communication problem here. In the next issue, we'll try to sort it out.
Authors
Bill Gaver is professor of design and co-director of the Interaction Research Studio at Goldsmiths, University of London. His research on design-led methodologies and innovative technologies for everyday life led him to develop an internationally recognized studio bringing the skills of designers together with expertise in ubiquitous computing and sociology. [email protected]
Kia Höök is a professor of interaction design at the Royal Institute of Technology (KTH) and works part-time at RISE SICS. Her research on interaction concepts including social navigation, seamfulness, somaesthetic design, and intermediate knowledge has been significantly influential on the application of theory to design practice within HCI. [email protected]
Copyright held by authors
The Digital Library is published by the Association for Computing Machinery. Copyright © 2017 ACM, Inc.
Post Comment
No Comments Found