Sensing at the Periphery
My thesis research focuses on exploring the possibilities of dynamic media design to facilitate more human and multi-sensory, and therefore, a more natural way to access information and to communicate.
In my thesis, I researched the historical, theoretical and psychological aspects of communication and technology and developed six case studies to determine how dynamic media design can incorporate the human senses in modern communication.
Major Thesis Project: "Drawing by Emotion"
â€œDrawing by Emotionâ€ uses the whole set, including the machine itself, the motion of the machine and the visual result as a process to transform vocal emotion cues to visual emotion expression.
The entire process combines human senses and a system installation to create a visual expression of emotion. Overall, in â€œDrawing by Emotionâ€, the interÂ¬action between human and machine, can be treated as part of Human-Robot Interaction or Human-Computer Interaction. Usually, the function of robots is to extend the range of the physical capacity of humans, especially in dangerous or extreme conditions. The basic goal of Human-Robot Interaction is to develop principles and algoÂ¬rithms to allow more natural and effective communication and interaction between humans and robots. â€œDrawing by Emotionâ€ seeks to elevate the process of interactivity. It envisions a way of manipulating a robotic machine that is capable of reacting to human voice that carries emoÂ¬tional cuesâ€”a human wayâ€”rather than a command-line interface. Its function would expand and integrate human senses.
There are several layers to the main elements of the project â€œDrawing by Emotionâ€:
When I designed each component of the project, the central question was: How could the machine express emotion? And how can that expression be accepted by people as an emotional?
Past experience of a meaningful object can easily wake up memory and then connect a person to a new meaning. Metaphor: The term of â€œmetaphorâ€ describes an expression of speech â€œin which a word or a group of words is transferred out of its actual context of meaning into another, without there being a direct simile between the descriptive term and that which it describesâ€(Duden)
â€œThe process of metaphor-creation mirrors the cognitive development of humanity. With the aid of metaphors, people are ability to find con nection in thought processes and bring areas of experience into relationship with each other.â€ (Jurgen Fritz)
Metaphor connected with memory. It connected the past experience and new experience to become a brand new unit and convert unfamiliar, unaccustomed to familiar and accustom. The use of metaphor is a process of transferring and connecting from the past to the present toward the future.
â€œDrawing by Emotionâ€ selects five object metaÂ¬phors to represent five basic emotions: Angerâ€”a hammer; happinessâ€”a spinning top, sadnessâ€”a IV set; neutralityâ€” a weaving loom; boredomâ€”a pendulum.
The reason for selecting those metaphors is based on my own and to common experience. The image of hammer crushing things, reminds me of the emotional statement of anger. The pleasant memories of spending hours playing with a spinning top as a child influenced my choosing a spinning top for a happiness machine metaphor. When I was in the hospital with a fever, I experienced how infusion is a long, painful process. That is the reason an IV set was chosen as the sadness machine metaphor. The motion of a clock pendulum always remains at the same speed and rhythm, which leaves a person feeling very bored. The mode and pattern of a moving loom leaves a neutral impression.
When I wake up my own memories for selecting metaphors, there are design questions: when we design what is the balance between â€œnewâ€ and â€œoldâ€? How to keep people feeling the ideas are fresh, but not overwhelm them? In his book, On Intelligence, Jeff Hawkin observes:
â€œBrains like familiarity, but they get bored. They are genetically programmed to want to discover new patterns; you donâ€™t want it too new because that seems dangerous. You want it somewhat familÂ¬iar and somewhat new. Somehow new and old at the same time gives the best design. If a design is so new that people canâ€™t relate to it, then they reject itâ€¦. you want it to be just slightly different, enough that people say, Oh, thatâ€™s coolâ€.
How does drawing specifically express emotion? Which drawing, which visual element, could represent which emotion? Not only does what I draw matter, but how I draw it matÂ¬ters too. So two parts need to be taken into consideration: the drawing process and the visual result.
The Drawing Process
What matters in the process of drawing are both motion and the tool used in making the mark. The speed, the force, the rhythm, the fluency and the loudness; each aspects of the movement all matter in communicating the meaning and expression of an emotion.
The expression, anger is based on other related words, for example, choler, conniption, infuriation, quick short temper, all of which help me to visualize the image. The basic movement of the anger machine is hitting. The variables are the height from which hitting begins, the speed of hitting, the rhythm of hitting and the force (intensity) of hitting.
The expression, happiness, is based on other related words, like delight and cheer. The basic movement of the happiness machine is circling. The variables are the size of the circling, the speed of circling, the rhythm of circling and its fluency, plus the width of the brush and the number of brushes.
The expression, neutrality, is based on other related words, such as disengaged and inactive. The basic movement of the neutrality machine is making straight lines. The variables are the number of strokes.
The expression, sadness, is based on other related words, such as dejection, dismal and blue. The basic movement of the sadness machine is dripping. The variables are the starting height of the dripping and the frequency of the dripping.
The expression, boredom, is based on other related words, such as lethargy and monotony. The basic moveÂ¬ment of the boredom machine is short repetitive stroking. The variables are the speed of stroking.
The Visual Result
Perception of color is the single most strongly emotional part of the visual process. Color has strong affinity to emotion. Color also carries symbolic meaning and associaÂ¬tive meaning.
There are many color theories. Relevant to my project is according to Donis A. Dondis who states that color has three dimensions which can be defined and measured: hue, saturation and brightness. In â€œDesign by Emotionâ€, I chose colors based on the general knowlÂ¬edge that bright colors often express excitement, while duller or darker ones can express relaxation, depression, sleepiness, or other low-key emotions. In the system, red, yellow, black, blue grey and grey represent respectively anger, happiness, neutrality, sadness and boredom. Each of the colors, except grey, which express boredom, has five different levels of hue, saturation and brightness to express different degrees of each emotion.
Each shape has its own unique character and characterÂ¬istics and meaning, a great deal of meaning is attached to each one. We get meaning in several ways, some through association, some though arbitrary attached meaning and some through our own psychological and physiological perceptions.
A splash shape, has qualities in irregularity, scalability, can express an intense emotion. Therefore this shape is used for representing anger in the system.
Curved lines are lively and suggest energy, they can express a pleasant emotion, so curved lines easily represent happiness in the system.
Straight lines convey neutral feelings, so useful for representing neutrality in the system.
Dots are delicate, tiny, like rain drops. They can express a â€œdownâ€ feeling, represent sadness in the system.
Strokes are monotonous and represent boredom in the system.
Pattern conveys multiple levels of information. One level combines the tools and the force of drawing. For example, in work with sharp, percussive strokes, like a Vincent Van Gogh painting, the brush strokes always give the painting an anxious tone.
The neutrality machine in â€œDrawing by Emotionâ€, uses sharp pencils and consistent force.
The anger machine uses a block of wood, a hammer and powerful pounding.
The happiness machine uses soft brushes and rapid light circular motions.
The sadness machine uses gentle dripping water.
Another level of pattern is connected with moveÂ¬ment. Pattern is a representation of motion on the timeline. The layer of patterns reveals the movement and its attached meaning.
Movement in a visual representation, is described as the compositional tensions and rhythms in visual data, even when what is being seen is actually fixed and unmoving. The layers of meaning in the drawing (pattern) are indiÂ¬cated by the intensity, the size, the speed and the length of time expended by the person making the drawing. From that, a great deal of emotional information is conveyed by the visual itself, the self-recorded visualization. The degree of emotion is conveyed in patterns related to movement.
Project Conclusions and Findings
In â€œDrawing by Emotionâ€, I have researched emotion recognition and especially focused on prosodic emotion recognition. I also have analyzed each step from formalizing of the concept through every step to the final interaction. Each design decision in â€œDrawing by Emotionâ€ is aimed to ensure that the project works well to express
Emotion is a notoriously hard concept to define. Even more difficult is to design for emotion. But that is crucial for designer to sense the right signal of emotional expression. Among many cues that can be treated as emotion recognition, vocal cues present high level of recognition. I selected widely accepted the classification of emotions based on known research refer to paper â€œEmotional Space Improves Emotion Recognitionâ€.
â€œDrawing by Emotionâ€ is a three steps process of representing five basic emotional states: anger, happiness, sadness, boredom and neutrality. The first step is object metaphorâ€”the right metaphor will reach a place of common knowledge that will help people â€œgetâ€
the emotion. The second step is the mechanics of the metaphorâ€”
the variables of motion, including the speed, the rhythm, the movement trends, the sound, etcâ€”all these can imply that emotion. The third step is visual resultâ€”from color, shape, pattern and self-record movementâ€”all these aspects represent emotion as well.
â€œDrawing by Emotionâ€ is an on-going project.
As soon as the prototype is fully functional, I plan to do user testing, particularly the input of users emotional expression vis-a-vis an output in the form the machineâ€™s â€œemotionalâ€ behavior.