Lev Manovich on Mon, 29 Sep 1997 20:58:45 -0700 (PDT) |
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
Syndicate: Jump over Proust: Toward Multimedia Writing |
Lev Manovich http://jupiter.ucsd.edu/~manovich Jump over Proust: Toward Multimedia Writing How does fictional, critical, theoretical, and historical writing can take advantage of all the multimedia capabilities offered by the Net? How to write about ideas using images, animation, sound, video and 3D worlds? How to communicate complex concepts while also being able to employ traditional rhetoric functions (seduce, convince, scare, inspire) using multimedia? How to allow the user not just to be simply a "co-author" (which is what the ideologists of interactivity naively aim at) but rather to take him/her "inside" the mental space of a text, inside the thinking process of another subject? How to think through multimedia? What are the historical precedents for "multimedia writing" in cinema, book design, theater, concrete poetry? [1] In my attempts to deal with these questions I was inspired by certain filmmakers who appear to be obsessed not simply with using cinema as a medium to convey ideas and arguments (which is what conventional documentaries are supposed to be doing) but rather as a medium capable of presenting the very process of thinking. Among these filmmakers I would single out Eisenstein, Marker and Godard. The first tried to formulate the notion of intellectual montage and planned to film Marx's Capital. Already at the end of the 1920s he was predicting that in the future philosophy and history will be presented as films: "The proclamation that I'm going to make a movie of Marx's Das Kapital is not a publicity stunt. I believe that the films of the future will be found going in this direction (or else they'll be filming things like The Idea of Christianity from the bourgeois point of view!) In any case, they will have to do with philosophy...the field is absolutely untouched. Tabula rasa." [2] Marker showed that cinema can be used to construct intellectual essays, the essays which associatively move from idea to image, and an image to another idea (for instance, in his recent "Level 5"). And Godard has already explored computer multimedia's new language in his films from the 1960s onward by systematically juxtaposing moving images, non-realistic sound and graphically presented texts; more recently he created a true multimedia masterpiece, an essay which takes us along his thinking process while using every multimedia code available: pages of text, still images, moving images, voice and sound ("JLG by JLG. Self-Portrait in November"). Not only to convey complex ideas through multimedia, but to take the reader along the process of thinking -- this is the challenge of multimedia writing. The use of a computer as writer's tool can only be justified if we can evolve more subtle, more complex and more precise ways of conveying what it means to think, of how it feels to move from one association to the next, from one memory to another, from one insight to the next. Only when we will give justice to the common view of a computer, which accompanied it from its very beginnings half a century ago, as a model of a human brain. A machine which has memory, which can store words and images, which can search and match, which, most importantly, can link, i.e. associate -- even if it is not a human mind, it has most of the functions we, humans, perform when we think. Therefore, we should be able to use a computer to portray human thinking in a more precise and engaging way than literature and cinema have already done. To do this is the challenge of multimedia writing. Earlier in this century, Proust, Nabokov, Joyce and other modernist giants came up with new techniques to represent our mental life: thinking and remembering, forgetting and repressing, formulating concepts, moving between the sensorial world outside and the mental world inside. Literature became the best mirror for the modern psyche, achieving highest fidelity in relation to our inner world. But other arts did not match its achievement. Cinema, multimedia's main precursor, is particularly disappointing in this respect. By and large, its language followed 19th century novel, rather than trying to match -- and go beyond -- the microscopic view of human inner experience recorded by Joyce, Nabokov and other moderns. This is particularly surprising, given that the first theoretical text on cinema -- Hugo Munsterberg's The Film: A Psychological Study (1916) -- proclaimed that the essence of the new medium lies in its ability to reproduce, or "objectify" various mental functions on the screen. According to Munsterberg, "The photoplay obeys the laws of the mind rather than those of the outer world." [3] In a provocative analysis, Munsterberg correlated the main cinematic techniques to different mental functions such as attention and memory, one-to-one. For example, in the close-up, "everything which our mind wants to disregard has been suddenly banished from our sight and has disappeared," analogous to how our attention selects a particular object from the environment. Similarly, the "cut-back" technique objectifies the mental function of memory. Yet, despite this promising analysis made by Munsterberg early on, cinema hardly took up the challenge of being a mirror of mind's operations. In my view, the only real systematic attempt in cinema to do this has been Godard's recent work, such as already mentioned "JLG by JLG" and also the majestic and monumental "Histories of Cinema" (which currently can be seen in Documenta X in Kassel). In the latter, Godard uses new cinematic techniques of his own invention in order to portray thinking process more accurately. For instance, he often superimposes 2, 3 or more images which gradually fade in and out, but never dissappear completely, staying on the screen for a few minutes at a time. It is as though these are ideas or mental images floating around in our minds, coming in and out of mental focus. Another technique involves replacing one image by another not through a cut or a dissolve but through a repeated oscillation, with two images flickering back and forth over and over, until the second image finally replaces the first. This technique can be interpreted as an attempt to represent mind's movement from one concept, mental image or memory to another -- the attempt, in other words, to represent what, according to Locke and other associationist philosophers, is the basis of our mental life -- forming associations. Yet, along with these innovative techniques which would certainly please Munsterberg (who, accidentally, was a professor of Psychology at Harvard University) by being visual equivalents of mental operations (or shall we say video-temporal equivalents, since time obviously plays a crucial role in Godard's techniques) Godard also often "capitulates" to cinema's more conventional way of narration: showing reality (here, a person thinking) from a third person point of view, i.e. from the outside. In "Histories of Cinema," we repeatedly see close-ups of a book page, or Godard himself, standing next to a bookshelf, getting a book, reading a sentence or two; or, finally, Godard sitting at a table and typing or writing. Perhaps these can be thought of as being equivalents of "establishing shots" in traditional cinematic narration: Godard's shows us his location (i.e., his mind) from the outside, so to speak, before taking us inside. Perhaps these images can be also interpreted as challenges to the conventional cinema and its extension, computer multimedia -- lets focus on intellectual life, on human mind rather than external actions and stories. So far a computer, despite his persistent association with a human mind, has served as even worse artistic mirror for our mind than cinema. This is strange given the fact that while only Munsterberg and few others made a connection between human mind and cinema language, in the case of a computer making similar connections became the research focus of a number of new fields, enormously successful, fields which keep expanding more and more: artificial intelligence, cognitive psychology, neuroscience -- in short, a whole set of disciplines grouped together under the name cognitive science, whose ultimate purpose is to map the language of the mind and the language of a computer one into another. While the attempts of artificial intelligence to simulate human mind have met with some limited success in such areas as parsing human speech, understanding stories, planning actions and interpreting images, the reverse problem -- the cultural problem -- using a computer to represent human mind in all its complexity and specificity (i.e., modeling not just the rational- computational part, as in artificial intelligence, but the phenomenological whole), pushing beyond what arts has accomplished so far -- was hardly even raised. Obviously, current language of multimedia -- presenting a user with a page containing a small number of links leading to other pages -- is hardly an adequate mirror of our mental life, or how we think, remember, plan, make connections. At present, software tools themselves are more revolutionary than multimedia applications they are used to design. They are better artistic visions of our inner life. Relational databases; pointers; control structures ("if... than," "case," etc.); object-oriented programming -- these and other programming concepts point towards potentially complex, dynamic and rich cultural representations of human mind. Even such seemingly trivial concept as a hierarchical file system is already more suggestive than the typical pages with hyperlinks which are being served to us in the 1990s under the slogan of "new media." Whatever it may involve, human thinking is certainly more like a computer program under execution (which involves translating between a hierarchy of computer languages, writing and reading data, keeping track of a current place in a program, clearing space in memory for new data and so on) than a set of pages linked by hyperlinks. To bring this new level of complexity, already achieved in software design, into the realm of cultural representation -- this is the challenge of multimedia writing. To do this, we need to be looking both at best cultural achievements in "mind modeling" -- Proust and Nabokov, Joyce and Godard -- and at the concepts of computer science, at the structure of computer hardware and software. Only when our multimedia texts will do justice both to the complexity of the machines used to compose and distribute these texts -- computers -- as well as to the complexity of what it feels to be a human being today: to think, to reflect, to carry the burden of human cultural history and of never before available amount of information and news from around the world, to interact with artificial minds of computers and with minds of other humans -- and also, as always, still to respond to the physical environment outside, the presence of others, to light, touch, and smell. In short, to be human, to reflect and to exist, to be inside and to outside at the same time. To represent this uniquely human, embodied thinking -- this is the challenge of multimedia writing. NOTES 1. In chapter 2 of my Ph.D. Dissertation, I explore many modern attempts to come with the techniques to represent ideas and logical arguments and to think through images. Lev Manovich, "The Engineering of Vision from Constructivism to Virtual Reality" (University of Rochester, 1993, unpublished). The historical material used in this text is drawn from this chapter. 2. Quoted in Annette Michelson, "Reading Eisenstein Reading 'Capital'," October 2 (1976): 28. Einstein's theory was not an isolated development. Many in the artistic left of the 1920s shared a similar belief in the cognitive power of new visual forms such as montage. In the late 1920s Alexander Rodchenko promoted the use of montage sequences in graphic design and, like Eisenstein, he saw montage as being equivalent to "dialectical" reasoning. In this formulation, an individual image corresponded to a single concept, and thinking was thought to be provoked when a number of images were juxtaposed in a series. Walter Benjamin's notion of "dialectical seeing," central in his unfinished Passagen-Werk project, also depends on montage, but within a single frame, so to speak. "Dialectical seeing" was conceived by Benjamin as a way to grasp the forms of the present by looking to the past and to the future in the same instant, juxtaposing them in the same mental image. 3. Hugo Munsterberg, The Photoplay: A Psychological Study (New York: D. Aplleton & Co., 1916), 41.