By Toby Bottorf
essay
The original attraction was probably based as much in mutual suspicion as anything else: the allure of the other. Computation and aesthetics hooked up and we are the bastard offspring, equally at home (and equally alien) at technical schools and art schools. But this fledgling discipline, dynamic media, is growing up self-assured and outgoing. Our field has helped improve the relationship between engineering and aesthetics by building a way to work, and a way to talk about the work, that includes them both.
From the outset, we have had to navigate irreconcilable differences in how to talk about this stuff. The early days of dynamic media (or interaction design) were governed by an engineering mindset, with little room for an arts vocabulary. While the basics of interaction design were being invented, usability was far more important than expressiveness. The most accessible entry point to interface design from the usability perspective remains Steve Krug’s Don’t Make Me Think. And yet, that represents only one valuable perspective or a phase our field has had to go through. Because what if we actually do want to make people think? What if we want people to engage with the interface consciously? These are legitimate artistic intentions.
Happily, engineering made space for art through its own achievements. Interfaces quickly became predictable in essential ways, and we developed a process of standardizing the rules of interaction into design patterns. Good. We have settled on a number of consistent approaches that cast in clearer relief the gestures and interactions that deliberately break the rules. The things that are unexpected can be assumed to be intentional now, not mistakes. In general, we can do more things wrong because what’s right has become agreed upon and familiar.
The inappropriateness of art in interfaces
No matter how cool your interface, it would be better if there were less of it.
— Alan Cooper, 1995
Compare this quotation to the feeling expressed in Beatrice Warde’s 1955 lecture (famous to graphic design students) arguing that typography should be a perfectly transparent vessel for content, a crystal goblet:
…it is mischievous to call any printed piece a work of art, especially fine art: because that would imply that its first purpose was to exist as an expression of beauty for its own sake and for the delectation of the senses.
Years apart, Warde and Cooper are worried about the same thing, that the medium might have some intrinsic message in it, that the designer might be the author of something. I find it reassuring to see old precedent for more recent anxieties. It suggests our concerns aren’t new nor bound tightly in a specific field of work. The fact that we consider these sentiments dated and only sometimes appropriate reflects how far we’ve come. Rules are for novices, whether novice people or entire disciplines, and we can confidently say that in much recent and ambitious work they don’t apply. They don’t apply to work that is mainly a kind of storytelling, though they still matter to the design of tools. But even there, the tools that we make out of dynamic media are increasingly a new kind of tool — they are services, not products. A well-designed service has to be more than just functional, it has to have an emotional component to it. User goals are not just behavioral, but affective too. As Donald Norman, stalwart of the usability perspective, puts it — attractive things work better.
I would go further and say challenging things are more interesting. Art is not user-abuse. The audience for art isn’t a “user.” Artists can and should summon the power of dislocation and disorientation as some of the most potent ways of focusing attention. These are ways of making people mindful of aesthetics, aware that they are looking. More challenging and startling experiences are called for.
These days user-abuse isn’t about interfaces anyway, it’s about EULAs (End-User License Agreement) and the legal terms of our relationships with digital services. Design is free to be challenging to the audience because we have a general consensus of interface patterns. Art was only ever a problem while the engineering was still unresolved.
I think this family squabble between art and engineering is over. It’s over because we realized there never was a conflict in the first place. The false Either-Or choice is answered with Both. We shouldn’t abuse the user, but we shouldn’t stop at just mitigating UI harm either. We are now as a field able to talk like architects about both the programmatic needs and the expressive intent. We trust that the stuff we build will stand. Now that we have an established set of interaction patterns, our ambitions move to emotional content and storytelling. We have mostly solved the engineering challenges — until we create new ones. Although, ask an architect, and they’ll admit that all great buildings leak.
So, on to creating new engineering challenges. We have tools and technologies at our disposal that seem to be capable of more than we have the imagination to invent. More art is desperately needed.
One way to aim for more surprising work is to include more disciplines as influences and sources of inspiration. Not to be dismissive, but we are not satisfied with just a revived Renaissance unity between art and science (no offense, Leonardo). We have not brought together something that was split in two. Rather, we’ve come to recognize hybridity as a defining quality of our discipline. We’re not a stable hybrid of different fields; we are all about the ongoing mixture of ideas. We stole from architecture when we sought models for navigable information structures, we enlisted anthropology to help us get better at contextual research and understanding users and cultures, and as long ago as 1991 Brenda Laurel proposed that we think of computers as theatre. Come on in. It seems we’ll welcome anyone.
Design and Everything Else
As the boundaries between different disciplines blur and dissolve, how do we resolve their different work methods and sometimes incompatible criteria for critique? I would bet on more and more disciplines becoming relevant to design, because ours is a restless and promiscuous discipline. Design-thinking values research and synthesis. It is outward-looking. But we need some limits. We need to be able to study and practice a field that doesn’t aspire to be about everything.
This calls for a curriculum that is both more fundamental — based in things of enduring value — and more modular and flexible. The fundamental is a basis in practice and craft, though not all craft is analog. We need craftsmanship in any number of specific areas. The flexibility calls on design-thinking, frameworks for work and critique across these different areas. Our domain easily includes animation and video, lighting and sound design, products and environments. Not only can we make stuff out of more and more responsive and dynamic materials, we can also make stuff about more personal and social experiences.
In 2005, the first year I was a critic at the Dynamic Media Institute, I was disoriented by the thesis project demonstrated by Lynn Faitelson (MFA 2005). Her project combined video, audio, and light effects mediated by the actions of the audience, and projected onto semantically loaded objects and materials. I asked whether I ought to approach this as a piece of interactive art or as a sculptural installation. The answer to that question told me how to handle my disorientation.
This challenging work also revealed to me a perennial difficulty in this field: it is hard to make deeply personal work, because the “person” in the art is the user-audience-participant and not the artist. In that regard maybe Warde and Cooper are not as wrong as they often seem to me, so long as we update what we think transparency means. Not necessarily something we see through, but something we can see into deeply. We need to design for open, unintended uses — paradoxical as that seems.
As the work gets more expressive, social and multi-sensory, its need to be self-exemplifying doesn’t go away. There is still a need for it to behave as an interface and tell you how to operate it. Through more artful means, this work still must direct the user’s attention toward their role and agency in the experience. If one axiom of software design is to build things under the assumption that nobody reads the manual, for more artistic work we should ensure that the audience’s experience includes their own process of writing the manual.
For the audience, discerning the relationship between cause and effect and understanding the feedback loops in the system are often a large part of the experience of interactive art. That offers an opportunity for deep engagement and self-reflection. It also highlights perhaps the greatest pitfall in our discipline: It is never enough for this work to just be a puzzle that the audience needs to solve. The most common trait of mediocre work is that the experience of it proceeds like this: mysterious, challenging, mysterious, inviting, mysterious, a sudden aha! and then banal. It cannot be like a punch line, dependent on the shock, and effective only once.
Good work maintains a level of elusiveness and invitation. It can succeed by never fully giving away its rules. If it does that in a way that isn’t arbitrary or hostile, it isn’t user abuse. Even better, it can continue to stimulate and delight (or challenge and disorient) after its rules for operation have been understood. There are abundant ways to achieve this. Sensory and social experiences are their own rewards, and games offer the pleasures of play and increasing mastery. More and more, we see these sensory, social, and playful experiences as being what our work is about.
Criteria for critique
From engineering we get one of the rules of critique for this kind of work: can the audience/user discern the feedback loop? That is, can I tell what is the relationship between input and output, between causes and effects? In simple interfaces, the feedback loop is usually one action, one effect, to confirm that the user action has been detected. For dynamic media, the loop is larger and often based on many input variables, like social behaviors.
A similarly analytic lens for critique is to evaluate projects as they are, giving little weight to intention. An art critique allows for investigation into motive, but I believe that is mainly useful as critical and formative feedback, driving the next phase of work. Final critique, such as I’ve enjoyed at many thesis defenses, should not be influenced much by motive. The work is finished. WYSYWYG. It either works or doesn’t.
If critique allows the question “does it work?” then the critic must approach the work as a user. Ideas and historical references (designing for designers) have to be evident in form, or they’re irrelevant. To some extent, we have to momentarily not know about the historical precedents in form and ideas. In effect, critique is a little like testing. That makes a form
of research.
What do we mean by research in academic work, in art, and in professional practice? These are three different perspectives that come together at DMI, and each recommends different and complementary things as worthy of research
- The academy wants us to understand our history well, place our work in a historical context, and advance the knowledge base for the field.
- An art program wants our research to generate many experiments with materials and our own selves, our motivations and personal mythologies.
- Professional practice wants us to research the context of use, and to test the usefulness and fit of prototypes with users.
All three encourage us to turn our research into something: to invent new things as well as new methods and tools for making things. The quality of research, and its value to the field depends on our documentation of process: our research into aspects of a problem and the tying together of many explorations into a coherent subject or approach. Admirably (and dauntingly) DMI asks that students defend their work in a written dissertation. They need to be able to tell a story about the entire body of their work. It reveals the methods taken, the personal applications of design-thinking to the craft of making stuff out of a broad range of different kinds of media. The collected dissertations capture DMI’s state and evolution of the art. This body of work suggests a way to address the relationship between design-thinking and practical craft, and engages the crucial question: What is the right level of media-specificity in a curriculum?
This question is proxy for another scary one: Do designers need to know code? My answer, simplified, is that design students definitely do. Know how it works, know what it does well and easily, know how to get help. We need to know how to make the things we design, or at least how they are made. There is nothing particular to dynamic media in this. I believe generally for all design disciplines that a deep grounding in craft, in how things are made, is essential. It leads to richer, more controlled, and more imaginative work. Before design was a profession, it was the quality of invention in a number of different professions: type designers were people who cut metal or stone, graphic designers were printers. Bradbury Thompson, one of my favorite designers, did work that was inseparable from the ink-and-paper potential of CMYK printing. Good design is not just expression of ideas: it’s made of materials, whether they be physical or digital, time based or space based. The same connection between ideas and craft that you find in the analog world of print is evident in projects where the material is code. Carolin Horn’s project from 2007, Anymails shows the same clear connection between ideas and the means of production. In this great project, emails are manifested as microbes, with a rich vocabulary of behaviors. The work expresses well what the medium affords and outlines its constraints. (And, yes, Carolin got help with the Flash and Processing code).
Art made out of code can be abstract, symbolic, or representational. Purely abstract art plots curves and shapes from algorithm. It is usually static, and not derived from any data set. More interesting and recent experiments take advantage of data sets that are large and dynamic. More and more data are becoming available as sensors grow in capability and ubiquity. The process of data-capture and display (the simple input/output model, still) leads to an incredible array of possible investigations. Colin Owens’ Shape Mix, from 2009, presents a focused investigation into the visual display of captured audio. This project goes beyond visualizing sound, investigating the uses of metaphor and the development of an extensible visual and behavioral language.
We judge this work on how well it shows us something new, or helps us see other things in new ways. In being about ideas and based in sensory experience, it aims, as design always must, to be smart in its beauty and beautiful in its intelligence.
Toby Bottorf is Principal of Digital Design, at Continuum and visiting faculty at the Dynamic Media Institute