Thinking Inside the Box

By Evan Karatzas, MFA ’05

essay

This essay originally appeared in The Experience of Dynamic Media: Works from the Dynamic Media Institute at Massachusetts College of Art and Design, 2006 – 2010 published on the occasion of DMI’s tenth anniversary.

Using System Logic as a Conceptual Tool

System logic lies at the center of interaction design. It establishes the operating principles of the experience and serves as a governing body that connects the actions of participants with the how, when, what, and why of the system’s computation and output.

Logic can and should play a central role in the conceptual development of virtually all interaction design. It is as essential to effective interactive experience design as aesthetic choices, interface, and content and can be used as a conceptual tool to push interaction and experience to new levels.

Complex formulas and flow charts often come to mind when we think about system logic and these forms can play an important role in expressing logic. But we’re talking about something else — a different way of looking at interaction and computation. To think of interaction design in terms of observable events and data is to see them as dynamic and interconnected elements which are in a constant state of change but which can be harnessed and exploited with intent.

Interaction As Computation

While system logic is generally expressed independent of the visual treatment of the work, it is impossible to consider it completely divorced from specific content, treatment, and method of interaction. In fact, the conceptual possibilities multiply when we explore our work with a sense of flexibility. Committing to a particular method of interaction or aesthetic too early can severely limit the true potential of the experience — as can indecision.

The designer is always challenged to make smart choices throughout the design phases of a project — to operate in that narrow, and sometimes uncomfortable, space between specificity and openness to radical revision. But too often, relationships between users and the system are not fully considered.

As designers, we’re accustomed to modulating between multiple perspectives as we shape our work. At a minimum we move between our perspective as an artist, documentarian, communicator or mediator, and an envisioned empathetic view of our users. These perspectives are fundamental, critical, and irreplaceable. But new conceptual opportunities appear when we augment these perspectives with a data-oriented, system-level view of the experience.

When Visual Exploration Is Not Enough

The Proximity Lab installation developed as the centerpiece of my thesis provides one example. From the beginning I had a strong sense of the type of experience I wanted participants to have — the installation would be human scale and the interface would be transparent, based on natural human interaction.

I had figured out a few important aspects of the installation; (a) the sole interface would be the real-time positions of participants as they moved around on the floor, and (b) the system would respond by way of projected image down onto the walking surface as well as with sound. But I struggled to understand what content would occupy the visual and auditory channels of the installation. Was this pre-recorded video footage, real-time footage from the gallery, or a collection of shapes painted by participants as they moved about?

Months of visual studies, sketches, and animated simulations ensued. Literally hundreds of hours of exploration later I was no closer to answering these questions than when I started. Something was missing and the solution would not come without looking at the problem from an entirely new perspective.

Frustrated by an inability to answer these content and aesthetic questions, I returned to the structural aspects of the project. Having clarified the experiential goal as visualizing social interaction, I was forced to consider and articulate precisely how various degrees of socialization would be recognized by the system. There would be no observer present to make these judgments and key them into the computer. The system would have to be able to make this determination on the fly.

Logic Provides the Answer

When I began expressing the conditions by which the system would rate the socialization of participants, the content, and eventually the visual language, came into focus. Proximity, or the amount of distance users create between each other, became the critical data point. I developed a formula for this socialization rating based on a set of detectable conditions including the time spent in close proximity to others and the choices individual users made. Terms like initiated, uninitiated encounter, and non-social departure — as well as a series of diagrams resembling dance step instruction — became ways of thinking about the system logic.

As these formulas and observable conditions were established, I started work on a screen-based prototype to demonstrate how the socialization rating would change as users moved. This prototype work included experimenting with a simplified calculation by representing personal space as a circle surrounding each user. It was easy to detect when these objects intersected and for how long and this provided another method of detecting close proximity and calculating socialization level.

The circles — an invention of necessity while developing the system logic, provided much more than a practical method for measuring social interaction. It forced me to view the experience from an entirely different perspective, from the view of the computational system if you will. This, combined with sound experiments I had been doing for another project, lead to a breakthrough. I discovered the visual and auditory language for the experience — modulating floral shapes as molecules that form a ring around users and which react and exchange when two or more users are in close proximity.

With further development, I realized that the content I was looking for — this prerecorded or real-time footage — was not necessary. Each participant would be assigned a unique color. The resulting distribution of colored shapes, based on the exchange of these molecules, provided a kinetic portrait of social interaction. This was the content and it spoke directly to the subject matter of the experiment.

Putting It Into Practice

This alternate view of experience and interaction can introduce new conceptual possibilities, perhaps even shift the trajectory of your work. Consider the following techniques designed to jump-start this process.

1. Think of interaction in terms of the data it produces

Your existing understanding of the work likely already includes some concrete interface elements and content. You probably have some idea of how this material will be delivered. The point of this exercise is not to discard or replace these components. The idea is to look beneath them at the structure of the system needed to facilitate the interaction. How are the various forms of user interaction recognized and recorded by the system? Do different interactions involve different sets of data?

2. Consider every component as a range of values

Reconsider your current understanding of how users can interact with the system. Broaden your view of how they might choose to participate. Do the same with your understanding of how the system will respond visually, audibly, or with new opportunities for participation.

Develop a matrix with a specific form of user interaction along one axis and specific system response along the other. Explore the extremes of user interaction and participation as well as system response. What values represent the highest and most active levels of participation? Is the absence of interaction accounted for in the system? If so, how is that represented and how will the system respond when this condition is observed?

If the interaction is designed to support multiple users, how might the system respond to a variety of interactions and participation levels? Return to your original concept and motivation. How can the system logic advance those goals? Should certain responses or participation levels be rewarded? Will you choose to illuminate the different types of user behavior or highlight their commonalities? Or is your goal to change their behavior, to nudge them toward a certain form of participation? Can they unlock a new level of system response or experience new content only available to users whose participation meets certain criteria?

3. Think of your system as event-driven

Consider the various ways participants can interact with each other and with the system within the construct of your designed experience. Abstract these as conditions and think about them as events that define a higher order of relationship between participants and
the system.

Instead of looking at individual actions and a correlating system response, put the granular user actions aside. Think about the experience in much broader, goal-oriented terms. Return again to your original concept and motivation. Define very broad levels of user achievement. Can certain combinations of user choices or patterns of behavior be defined and linked to more elaborate system responses?

That is to say, can the totality of a user’s interaction be recognized by the system? Perhaps this could alter or extend the relationship between users and the system? Think of this as an event that the system is watching for. What are the major events — separate from but connected to individual interaction — that the system is prepared to identify and respond to?

4. Consider and exploit the visibility of system logic

Think about how the disclosure of system rules to participants may affect the way they interact. Explore a variety of approaches and consider how they align with the experiential goals of your work. What happens if you withhold all instruction and prompting? How might this affect the way participants choose to interact as they investigate the system and try to understand its rules? Will it give users more opportunity to shape their own experience with the system? With other participants? How will it affect approachability and usability? Will users be more or less likely to participate in the absence of clear instruction and details about how the system works?

Explore the other end of the spectrum where everything is explained. How might this affect participation and the opportunity for users to experiment and explore? Is the disclosure of these rules a dynamic element that is revealed as user interaction unfolds? What is the proper balance between ambiguity and disclosure that will encourage and enable the type of experience that you seek and that participants will find fulfilling?

In Conclusion

Not all of the techniques discussed here will be applicable. Some will be more effective than others depending on the nature of your work, specific goals, and motivation. Most importantly, these techniques cannot replace experimentation, user testing, and direct observation. Prototyping your work early and often is critical to testing your assumptions and learning how users will really behave.

There is no substitute for thoughtful research, an iterative approach to development, and a constant critical examination of your assumptions. But combine these best practices with a multifaceted approach to interaction design — as an artist or communicator, from the vantage point of end users, and by adopting a personified view of the system — and there is no question that the conceptual and experiential possibilities of your work will expand.

There are many more questions to ask. The exploration of shared authorship and relinquished control are critical considerations. And of course, there is never a formula that leads predictably to creative solution. You must pose questions specific to your interests about your intellectual investigation and expressive goals, then explore them and allow them to branch into new directions and possibilities.

Evan Karatzas is founder of Proximity Lab and visiting faculty at the Dynamic Media Institute.