Name: Jan Kubasiewicz, Professor of Design
Maps and Mapping as Metaphors
Maps are made for different territories and different themes — real or imagined — not necessarily related to geography. As symbolic depictions of those territories, traditional maps were limited to a static, two-dimensional representation, whereas today’s maps may be multidimensional, dynamic and interactive. Regardless of the medium, maps are tools of reasoning revealing relationships between elements, properties, and concepts. Mapping creates associations between equivalent groups of qualities that are organized according to rules of correspondence based on a particular system. The word “map,” both as verb and noun, seems very appropriate to describe the essence of dynamic media.
This text, metaphorically speaking, is a concept map describing the territory of dynamic media and its pedagogy as practiced at the Dynamic Media Institute. It represents the humanistic approach to the study and practice of dynamic media with its central focus (as the humanities) on the creation and documentation of cultural artifacts. Those artifacts are human experiences mediated, or perhaps the better term would be “curated,” within the computational complexity of social communication.
Communicating in the language of dynamic media requires fluency in multiple “languages,” “dialects,” and “codes” that have traditionally been segregated into distinct disciplines. It demands a synthesis of multiple points of view on communication. At the Dynamic Media Institute we take advantage of the fact that each participant in the program brings a unique background and vision to the discourse.
Graphic designers and information architects, filmmakers and writers, musicians and programmers all offer diverse points of view and use the different “native” languages of their respective professional fields in describing the human experience of communication. To adequately address the multiple aspects of dynamic media requires a combination of these expert points of view, accomplished through a difficult dialog along the borderlines of multiple disciplines.
As the DMI program focuses on communication design, our notion of dynamic media is closely related to information and its broad range of meanings from information architecture to data visualization.
Furthermore, the notion of dynamic media is also closely related to motion — and therefore time. Motion, as integral to design, is considered a language of communication in our curriculum, and consequently “motion literacy” — the act of understanding how motion can be used to communicate more effectively — becomes an essential component of our pedagogy.
Lastly, dynamic media is related to interaction. The terms “interaction” or “interactivity” describe an interdisciplinary field encompassing those aspects of art, design, science, and engineering involved in bringing meaningful experiences to people. Interactive systems — human-to-computer, human-to-artifact, and human-to-human — mediate the process of communication and therefore augment the participant’s experience as well as the environment where communication occurs. Accordingly, in this text the mapping and discussing of dynamic media is arranged within the following groups of concepts: Design for Information; Design for Time and Motion; Design for Interaction.
Design for Information
The beginning of the Information Age was marked by the concept of “information” as defined by mathematicians and computer scientists to solve the problems of sending and receiving messages. As a concept, information was abstracted and extracted from the news, or image, or sound. Cybernetics — an interdisciplinary approach to the study of systems and structures of information born in the 1950s — considered information an abstract sequence of signs. Such definitions of information allowed then, and now, for comparisons and analogies between scientific disciplines as well as for finding parallels between science and art.
Dynamic media designers see the world as an information structure that communicates continuously. For dynamic media designers information relates very closely to the essential notions of communication design: knowledge, learning, language, perception, and many others. Consequently, the subject of information requires a unique and highly analytical approach from designers and design educators.
In everyday conversations we often use the words “data,” “information,” and “knowledge” interchangeably. However, they are not synonymous and, in fact, refer to different levels of information structure. In his multiple publications related to information design, Nathan Shedroff uses the following progressive sequence of terms to explain the hierarchy of information organization: “Data–Information–Knowledge–Wisdom.” [1] “Data is not information …” and “… information is also not the end of the continuum of understanding,” [2] This is a critical realization for communication designers.
Data represents only a potential to become information. It is context that gives data a structure that the human brain is able to process. Similarly, information is not knowledge. Only in the context of knowledge — a larger volume of highly organized information accumulated as a result of experience — can new information become “visible” or “recognizable.” It can then be processed, verified, and eventually become part of knowledge.
We have just described the process of learning — a change in information processing pattern that occurs as a result of experience. Learning, and subsequently knowledge, is a result of a communication process that can only be completed by an individual’s brain. The designer can help, but cannot complete this process for the learner.
Human communication is a process that “relies on artistic techniques, on inventions, on tools and instruments, that is, on symbols ordered into codes.” (Vilém Flusser) [3] Homo sapiens are born with some of these “tools and instruments” — or rather we adapt our body to become “tools” able to articulate symbolic languages. We invent these languages as well. And our ability to create stories is a powerful, primordial communication model.
In his overview of narrative, Roman Jakobson uses the terms “context” and “contact” to describe “… a physical channel and psychological connection between the addresser and the addressee enabling both of them to enter and stay in communication.” [4] We’ve already concluded that it is the context which gives data a structure that the human brain is able to process. The term “context” also refers to the complexity of technology and the various degrees of participation by multiple users.
The notion of “contact” is very relevant today. In contemporary practice of dynamic media design the term “contact” — or its contemporary equivalent “flow” (Mihály Csíkszentmihályi) — relates to the situation of full focus and involvement, usually within multi-sensory environments, where and when a person is entirely immersed in the process of communication and interaction. Today such experiences — mediated by interactive systems often in the background or peripherally — are capable of simulating the complexity of a true multi-sensory human experience.
Multi-sensory experience — as a model for dynamic media communication — makes a clear argument for multiple modalities of information. In his book Multiple Intelligences, Howard Gardner reveals several distinguishable preferences of individual competence and creativity. [5] This brilliant concept, so influential among educators, simply acknowledges the fact that people think differently. These types of personal preferences in learning — “intelligences” according to Gardner — also describe participants in any process of communication, including designers and audiences of dynamic media.
“An intelligence is a computational capacity — a capacity to process a certain kind of information — that originates in human biology and human psychology.” [6] Gardner identifies seven preferable learning styles: Linguistic intelligence (involving words); logical-mathematical intelligence (numbers and logic); spatial intelligence (visual and spatial thinking); musical intelligence (music and sound); bodily-kinesthetic intelligence (natural sense of body movement); interpersonal intelligence (involvement with others, understanding them); intrapersonal intelligence (self-reflection and self-understanding).
Mapping may be considered another form of natural intelligence. Among many strategies for communicating in the language of dynamic media, mapping seems to be a dynamic media designer’s natural way of thinking. It is a unique cognitive skill of finding connections among things, and making them visible to others who cannot see on their own. It requires both types of skills: the intellectual skill needed in research and theory, and the manual dexterity needed to translate concepts into visualizations. For users, mapping is a process whereby knowledge may be created, rather than revealed. It is a tool of the thought process.
Images are one means of representing information — different obviously from numerical and verbal description. Yet logic, not imagery, communicates the true intention. Application of logic tied strategically to visual form — in other words visual logic — creates a unique way of interacting with information for both designers and users. By viewing, reading, and scanning visual patterns — static or dynamic — and by selecting subjective paths through the content of maps and diagrams, users learn in their own unique way.
Visualization of information supports our intuition toward abstract thinking and therefore understanding. Understanding combines the rational and the emotional: the knowledge frozen in words and numbers, and the knowledge vested in sensory experiences. The best examples of visualization are always charged with imagination — and often with poetry. Visualization extends to discovery. These images can teach us something, as their forms seem inseparable from the information they convey and knowledge they create or reveal.
Researchers of all disciplines apply computational tools — including dynamic visualization and simulation tools — to “observe” and analyze the data in search of patterns and connections, often prior to defining scientific hypothesis. This is what some call “the fourth paradigm of science” (the term coined by James Nicholas Grey).
In processing and analyzing unprecedented amounts of data collected via sophisticated tools of observation, researchers are helped by designers — contemporary cartographers — who aid in developing systems of visualization and mapping to navigate large scale patterns of information, that not so long ago were totally invisible to us.
Design for Time and Motion
Motion is integral to design. It is considered a language of communication in the curriculum of the Dynamic Media Institute. Consequently, “motion literacy” — understanding how motion can be used to communicate more effectively — becomes an essential component of our pedagogy. The notion of time, intertwined with motion, is considered the organizing principle to which all other design elements must relate. It gives emphasis to the process of forming rather than form. The meaning of motion has already been explored within multiple disciplines of art and science. Various expert languages, codes, and dialects of motion have been developed in cinema, music, choreography, etc. Communication designers must learn these languages.
“Communicating in the language of motion involves issues of what is moving and how that something is moving. The how question refers to the kinetic form and its grammar defined by space and time dimensions.” [7] Kinetic “behavior” of typography, cartoon or diagram, contributes an additional layer of meaning to the objects that already convey messages expressed in their own native languages of pictures, words, or numbers.
One of the most spectacular historical examples of the dynamic media design process is a post-production diagrammatic storyboard for “Alexander Nevsky,” a 1938 film by Sergei Eisenstein, a Russian film director and one of the first theorists of the medium. [8] That storyboard is a timeline in which visual representation of the film components are precisely synchronized into a sequence of “audio-visual correspondences” including film shots, music score, a “diagram of pictorial composition,” and a “diagram of movement.” The “diagram of movement” represents specifically the camera work resulting in on-screen motion. Choreographed very precisely, in fact to a fraction of musical measure, this “diagram of movement” attests to how essential, for the cinematographer, was on-screen motion and its meaningful integration with all other elements of his vocabulary. The same challenge of integrating motion as a meaningful component of communication design remains the focus of research and practice in our curriculum.
Studying cinematic narrative is always a source of inspiration for designers. In its hundred-year history, the cinematic vocabulary has evolved into a complex, universally understood system of communication, a system capable of translating a multi-sensory human experience into a kinetic sequence of audio-visual events, where motion serves to integrate all other channels of communication.
“As the mind perceives visual, sonic, and kinetic information over a period of time, it continuously organizes discrete units or messages into a story, however abstract that story might be.” [9] A story is a sequence. It must have a beginning, middle, and end (after Aristotle), though (after Godard) they need not necessarily be told in that order. A designer’s awareness of two distinct timelines — one for the story, another for the storytelling — is therefore essential. Equally essential is the designer’s awareness of the “plasticity” of time, and consequently, the designer’s skill in manipulating time — real time, its representation and perception — through motion and sequencing.
Since Charles and Ray Eames’ “Powers of Ten” (1977) integration of motion and sequencing with information design has demonstrated a tremendous potential for contributing, through sequential visualizations, to various disciplines of science, economy, and education. [10] Similarly, there is no need to argue any more that dynamic visualizations seem to be the only practical solutions for managing complexity of large-scale information structures.
The history of dynamic visualization seems to accelerate. It is difficult to realize that widely accepted concepts and metaphors of dynamic visualization and interaction with data — such as transparent intersecting planes, infinite zoom, dynamic points of view, manipulablility, zero-gravity 3-D space — did not exist before Muriel Cooper presented the MIT Media Lab Visible Language Workshop projects to the TED conference audience in 1992. [11]
Of course motion has been an essential structural element in various forms of expression. The art of music provides extraordinary case studies of using the expert language of motion for the purpose of communication. In “The Seventh Door,” a documentary film on Peter Eötvös, there is a scene where three music conductors Pierre Boulez, Peter Eötvös, and David Robertson rehearse Karlheinz Stockhausen’s “Gruppen” — the composition for three orchestras placed to the left, centre, and right of the audience. [12] In this piece Stockhausen explores the spatial location of sound by separating music themes from orchestra to orchestra, and creating an effect of sound moving in space. In the documentary only the three conductors are rehearsing the composition. They communicate among themselves in a very precise language of gesture while reading the score and translating musical text into their own interpretation of the piece. Without other musicians, Boulez, Eötvös, and Robertson seem to have an intense conversation, yet they conduct without producing any sound.
Gesture, understood as motion that has meaning, has already become part of the vocabulary of communication design. Computer users widely accept the kinetic behavior of icons that move or jump a certain way, in order to tell us something. Through their “gestures” icons notify us, warn us, and prompt us to action. These behaviors are not formally codified, but perhaps they should become a convention, in order to be even more precise in “what” and “how” they communicate.
The success of multiple-touch displays has elevated gesture to a new role in interface design. “We’re just scratching the surface of multitouch” optimistically proclaims Jeff Han, one of the pioneers in designing and engineering multitouch technology. [13] It seems that gestural interface introduced in the well-known science fiction movie, “Minority Report,” became reality, and according to Jeff Han, perhaps in its better version. Better, “because purely gestural interfaces actually work very poorly. It’s been proven. The human body really needs that kind of tactile feedback.” [14] Multitouch technology delivers that tactile experience, and is now migrating away from stationary to mobile multitouch displays to be combined with other devices. Current popularity of the “Wii” gaming platform, iPhone, and iPad are precursors of things to come, where integrating the function of gesture and touch “may actually be more successful that each one on its own.” [15]
Design for Interaction
Bill Verplank asks three fundamental questions of interaction design while drawing his brilliant diagram in Designing Interactions. [16] The first is: “How Do You Do?” a question that relates to a possibility of an action the interface offers the user — “… you can grab a handle … or push the button.” The second is: “How Do You Feel?” a question that relates to feedback the interface gives to the user — “… feelings come from … the sensory qualities of media.” And the third is: “How Do You Know?” a question that relates to learning and understanding the interface — “… [the] map shows the user an overview … the path shows them what to do.” [17] Verplank’s questions focus on three essential areas of concern to dynamic media designers creating user experiences — regardless of the environment in which the interaction occurs. “Doing” means acting. “Feeling” means reacting to feedback. “Knowing” means learning and understanding the system.
The user interface is the front-end of an interaction. The back-end of any dynamic system of information — invisible to the user — is a database. Information architecture addresses the issues of structure and organization of information from the user’s point of view. By running hypothetical user scenarios, the goal of the information architect is to design appropriate information flows within the system — often very complex systems with multiple “touchpoints” and multiple modes of interaction.
Consequently, information architecture is part of the design process in almost all work developed at DMI. However, it should not be considered a discipline of design. Information architecture represents an approach to design that allows the designers to see the information flow in any design product. It can be applied to traditional communication design as well as to other design disciplines, since information flow defines not only digital interfaces, but also analog interfaces of objects, as well as services and environments.
Dynamic media designers see the world as an information structure that communicates continuously and persistently. Any human experience involves dynamic information flow, therefore communication process, therefore learning. Communication and learning are as inseparable from human experience as from time and motion. Even if we do not want to communicate — by not doing so — we communicate. “You cannot not communicate.” (Paul Watzlawick) But as communication designers you can always learn how to communicate more effectively in the language of dynamic media.
As DMI educators in this age of “tsunami of data” (Richard Saul Wurman) we choose to focus our curriculum on the role of dynamic media. In doing so we are helping ourselves and others to participate in the complexity of information in the world today — in learning and understanding, making informed decisions, articulating meaningful thoughts, creating, entertaining, telling stories and enjoying the experience of all of the above.
Jan Kubasiewicz is Professor at Massachusetts College of Art and Design
References
1.Nathan Shedroff, “Information Interaction Design: A Unified Field Theory of Design,” http://www.nathan.com/thoughts/unified/ (1994): 3.
2. Ibid.
3. Vilém Flusser, Writings, Andreas Ströhl, ed. (Minneapolis: University of Minnesota Press, 2002), 3.
4. Roman Jakobson, “Closing Statement: Linguistics and Poetics,” in Style and Language, Thomas Sebeok, ed. (Cambridge, Mass. 1960), 353.
5. Howard Gardner, Multiple Intelligences. New Horizons (New York: Basic Books, 2006).
6. Ibid., 6.
7. Jan Kubasiewicz, “Motion Literacy,” in: The Education of a Graphic Designer, 2nd ed., Steven Heller, ed. (New York: Allworth Press, 2005), 181.
8. Sergei Eisenstein, The Film Sense, Jay Leyda, ed. (San Diego: HBJ, 1975), 175.
9. Jan Kubasiewicz, “Motion Literacy and the Language of Dynamic Media,” in: The Language of Dynamic Media, (Boston: MCAD, 2005), 15.
10. Powers of Ten, dir. Charles and Ray Eames, 9 min. 1977.
11. Information Landscape, MIT Center for Advanced Educational Services VHS tape. 1994.
12. Peter Eötvös: The Seventh Door, Juxtapositions DVD, 2005.
13. Bryan Gardiner, “Jeff Han: We’re Just Scratching the Surface of Multitouch,” Wired Magazine, 26 August 2008.
14. Ibid.
15. Ibid.
16. Bill Moggridge, Designing Interactions, (Cambridge, Mass.: MIT Press, 2007).
17. Ibid., 127.