Sensing at the Periphery

Li, Mingxi
Completion Date: 

My thesis research focuses on exploring the possibilities of dynamic media design to facilitate more human and multi-sensory, and therefore, a more natural way to access information and to communicate.

In my thesis, I researched the historical, theoretical and psychological aspects of communication and technology and developed six case studies to determine how dynamic media design can incorporate the human senses in modern communication.

Major Thesis Project: "Drawing by Emotion"

“Drawing by Emotion” uses the whole set, including the machine itself, the motion of the machine and the visual result as a process to transform vocal emotion cues to visual emotion expression.

The entire process combines human senses and a system installation to create a visual expression of emotion. Overall, in “Drawing by Emotion”, the inter¬action between human and machine, can be treated as part of Human-Robot Interaction or Human-Computer Interaction. Usually, the function of robots is to extend the range of the physical capacity of humans, especially in dangerous or extreme conditions. The basic goal of Human-Robot Interaction is to develop principles and algo¬rithms to allow more natural and effective communication and interaction between humans and robots. “Drawing by Emotion” seeks to elevate the process of interactivity. It envisions a way of manipulating a robotic machine that is capable of reacting to human voice that carries emo¬tional cues—a human way—rather than a command-line interface. Its function would expand and integrate human senses.

There are several layers to the main elements of the project “Drawing by Emotion”:

Object Metaphor
When I designed each component of the project, the central question was: How could the machine express emotion? And how can that expression be accepted by people as an emotional?

Past experience of a meaningful object can easily wake up memory and then connect a person to a new meaning. Metaphor: The term of “metaphor” describes an expression of speech “in which a word or a group of words is transferred out of its actual context of meaning into another, without there being a direct simile between the descriptive term and that which it describes”(Duden)

“The process of metaphor-creation mirrors the cognitive development of humanity. With the aid of metaphors, people are ability to find con nection in thought processes and bring areas of experience into relationship with each other.” (Jurgen Fritz)

Metaphor connected with memory. It connected the past experience and new experience to become a brand new unit and convert unfamiliar, unaccustomed to familiar and accustom. The use of metaphor is a process of transferring and connecting from the past to the present toward the future.

“Drawing by Emotion” selects five object meta¬phors to represent five basic emotions: Anger—a hammer; happiness—a spinning top, sadness—a IV set; neutrality— a weaving loom; boredom—a pendulum.

The reason for selecting those metaphors is based on my own and to common experience. The image of hammer crushing things, reminds me of the emotional statement of anger. The pleasant memories of spending hours playing with a spinning top as a child influenced my choosing a spinning top for a happiness machine metaphor. When I was in the hospital with a fever, I experienced how infusion is a long, painful process. That is the reason an IV set was chosen as the sadness machine metaphor. The motion of a clock pendulum always remains at the same speed and rhythm, which leaves a person feeling very bored. The mode and pattern of a moving loom leaves a neutral impression.

When I wake up my own memories for selecting metaphors, there are design questions: when we design what is the balance between “new” and “old”? How to keep people feeling the ideas are fresh, but not overwhelm them? In his book, On Intelligence, Jeff Hawkin observes:
“Brains like familiarity, but they get bored. They are genetically programmed to want to discover new patterns; you don’t want it too new because that seems dangerous. You want it somewhat famil¬iar and somewhat new. Somehow new and old at the same time gives the best design. If a design is so new that people can’t relate to it, then they reject it…. you want it to be just slightly different, enough that people say, Oh, that’s cool”.

How does drawing specifically express emotion? Which drawing, which visual element, could represent which emotion? Not only does what I draw matter, but how I draw it mat¬ters too. So two parts need to be taken into consideration: the drawing process and the visual result.

The Drawing Process
What matters in the process of drawing are both motion and the tool used in making the mark. The speed, the force, the rhythm, the fluency and the loudness; each aspects of the movement all matter in communicating the meaning and expression of an emotion.

The expression, anger is based on other related words, for example, choler, conniption, infuriation, quick short temper, all of which help me to visualize the image. The basic movement of the anger machine is hitting. The variables are the height from which hitting begins, the speed of hitting, the rhythm of hitting and the force (intensity) of hitting.

The expression, happiness, is based on other related words, like delight and cheer. The basic movement of the happiness machine is circling. The variables are the size of the circling, the speed of circling, the rhythm of circling and its fluency, plus the width of the brush and the number of brushes.

The expression, neutrality, is based on other related words, such as disengaged and inactive. The basic movement of the neutrality machine is making straight lines. The variables are the number of strokes.

The expression, sadness, is based on other related words, such as dejection, dismal and blue. The basic movement of the sadness machine is dripping. The variables are the starting height of the dripping and the frequency of the dripping.

The expression, boredom, is based on other related words, such as lethargy and monotony. The basic move¬ment of the boredom machine is short repetitive stroking. The variables are the speed of stroking.

The Visual Result

Perception of color is the single most strongly emotional part of the visual process. Color has strong affinity to emotion. Color also carries symbolic meaning and associa¬tive meaning.

There are many color theories. Relevant to my project is according to Donis A. Dondis who states that color has three dimensions which can be defined and measured: hue, saturation and brightness. In “Design by Emotion”, I chose colors based on the general knowl¬edge that bright colors often express excitement, while duller or darker ones can express relaxation, depression, sleepiness, or other low-key emotions. In the system, red, yellow, black, blue grey and grey represent respectively anger, happiness, neutrality, sadness and boredom. Each of the colors, except grey, which express boredom, has five different levels of hue, saturation and brightness to express different degrees of each emotion.

Each shape has its own unique character and character¬istics and meaning, a great deal of meaning is attached to each one. We get meaning in several ways, some through association, some though arbitrary attached meaning and some through our own psychological and physiological perceptions.

A splash shape, has qualities in irregularity, scalability, can express an intense emotion. Therefore this shape is used for representing anger in the system.

Curved lines are lively and suggest energy, they can express a pleasant emotion, so curved lines easily represent happiness in the system.

Straight lines convey neutral feelings, so useful for representing neutrality in the system.

Dots are delicate, tiny, like rain drops. They can express a “down” feeling, represent sadness in the system.

Strokes are monotonous and represent boredom in the system.

Pattern conveys multiple levels of information. One level combines the tools and the force of drawing. For example, in work with sharp, percussive strokes, like a Vincent Van Gogh painting, the brush strokes always give the painting an anxious tone.

The neutrality machine in “Drawing by Emotion”, uses sharp pencils and consistent force.

The anger machine uses a block of wood, a hammer and powerful pounding.

The happiness machine uses soft brushes and rapid light circular motions.

The sadness machine uses gentle dripping water.

Another level of pattern is connected with move¬ment. Pattern is a representation of motion on the timeline. The layer of patterns reveals the movement and its attached meaning.

Movement in a visual representation, is described as the compositional tensions and rhythms in visual data, even when what is being seen is actually fixed and unmoving. The layers of meaning in the drawing (pattern) are indi¬cated by the intensity, the size, the speed and the length of time expended by the person making the drawing. From that, a great deal of emotional information is conveyed by the visual itself, the self-recorded visualization. The degree of emotion is conveyed in patterns related to movement.

Project Conclusions and Findings
In “Drawing by Emotion”, I have researched emotion recognition and especially focused on prosodic emotion recognition. I also have analyzed each step from formalizing of the concept through every step to the final interaction. Each design decision in “Drawing by Emotion” is aimed to ensure that the project works well to express
emotional statements.

Emotion is a notoriously hard concept to define. Even more difficult is to design for emotion. But that is crucial for designer to sense the right signal of emotional expression. Among many cues that can be treated as emotion recognition, vocal cues present high level of recognition. I selected widely accepted the classification of emotions based on known research refer to paper “Emotional Space Improves Emotion Recognition”.

“Drawing by Emotion” is a three steps process of representing five basic emotional states: anger, happiness, sadness, boredom and neutrality. The first step is object metaphor—the right metaphor will reach a place of common knowledge that will help people “get”
the emotion. The second step is the mechanics of the metaphor—
the variables of motion, including the speed, the rhythm, the movement trends, the sound, etc—all these can imply that emotion. The third step is visual result—from color, shape, pattern and self-record movement—all these aspects represent emotion as well.
“Drawing by Emotion” is an on-going project.

As soon as the prototype is fully functional, I plan to do user testing, particularly the input of users emotional expression vis-a-vis an output in the form the machine’s “emotional” behavior.

Project Gallery: 

Project Keywords:
Login | The Dynamic Media Institute and Massachusetts College of Art and Design are trademarks and all contents of this site are Copyright 2005 - 2012 unless otherwise noted. Writings and projects are property of their respective authors. Reproduction in whole or in part is prohibited without prior consent.