<nettime> FreezeFrame - an essay on music composition on the Internet

["Paul D. Miller" <anansi1 {AT} earthlink.net>]

Hello you all this is part of a longer essay, but basically I'm going 
to present this and some live software remixes of materiaal from 
various digital sound sources.  It'll be both a multi-media lecture 
and show how dj culture relates to digital culture etc etc Ken 
Jordan's book with Randall Packer "Multimedia: From Wagner to Virtual 
Reality" has become a classic on the history of multimedia with a 
broad historical perspective, and basically the article engages ideas 
about how networked environments interact with the composition 
process. The article first appeared as  a commision on the new music 
box website:

www.newmusicbox.org

check it out!

thanx,
Paul

Freeze Frame: Audio, Aesthetics, Sampling, and Contemporary Multimedia
by Ken Jordan and Paul D. Miller a.k.a. DJ Spooky that Subliminal Kid


Paul D. Miller's Preamble:

In an era of intensely networked systems, when you create, it's not 
just how you create, but the context of the activity that makes the 
product. Let's think of this as a hypothetical situation become real, 
and then turn the idea inside out and apply it to music -  operating 
systems, editing environments, graphical user interfaces -  these are 
the keywords in this kind of compositional strategy. During most of 
the spring of 2002 I was working on an album called "Optometry." I 
thought of it as a record that focused on "the science of sound - as 
applied to vision." Think of it as a kind of "synaesthesia" project 
navigating the bandwidth operating between analog and digital realms. 
"Optometry" was constructed out of a series of audio metaphors about 
how people could think of jazz as text, of jazz as a precedent for 
sampling  - of jazz as a kind of template for improvisation with 
memory in the age of the infinite archive. In sum, the album was a 
play on context versus content in a digital milieu using sampling as 
a "virtual band" of the hand. Flip the situation into the here and 
now of a world where file swapping and peer-2-peer bootlegs are the 
norms of how music flows on the web, and "Optometry" becomes a 
conceptual art project about how the "hypertextual imagination" holds 
us all together. Seamless, invisible, hyper-utilitarian... those are 
some of the words that describe the composition process of 
"Optometry." What's new here? In 1939 John Cage made a simple 
 statement about a composition made of invisible networks that was 
called "Imaginary Landscape." The piece was written for phonographs 
with fixed and variable frequencies (consider that there was no 
magnetic tape at that time), and radios tuned to random stations. The 
idea for Cage was that the music was an invisible network based on 
"chance operations."  As Cage would later say in his famous 1957 
essay "Experimental Music," "Any sounds may occur in any combination 
and in any continuity." The sounds of one fixed environment for him 
were meant to be taken out of context and made to float - think of it 
as audio free association, and you get the first formalist ideas of 
the origins of DJ culture. But what does this have to do with jazz?

In 1964 Ralph Ellison gave a discussion before the Library of 
Congress about writing jazz criticism. In it he discussed Henry 
James's fascination with Americaness - think of it as an echo of the 
Cage notion, and flip the code into a different cipher - you arrive 
at Henry James' critique of Americanness as "a complex fate." The 
Ellison lecture was called "Hidden Name/Complex Fate" and Ellison 
takes us on a journey through elements "absent from American life." 
In this text Ellison would flip the mix and build a template for a 
new kind of literature - that's the echo of "Imaginary Landscape" 
that intrigues me. He said: "So long before I  thought of writing, I 
was playing by weather, by speech rhythms, by Negro voices and their 
different timbres and idioms, by husky male voices and by the high 
shrill singing voices of certain Negro women, by music by tight 
places and wide spaces in which the eyes could wander..." Again, the 
invocation of an imaginary landscape made of the hyper-real 
experiences of living in a world made of fragments. That's what 
"Optometry" inherits. Think of it as a dialectical triangulation 
between the idea of being made from files of expression put through 
places that are not spaces, but code. Gesture is the generative 
syntax, but once the sounds leave the body, they're files. And that's 
the beginning...


1.

When computers communicate over a network, they do so through sound. 
Before information can be sent over wires run between computers, it 
must first be translated into tones. The composer Luke Dubois, of 
Columbia University's electronic music department, has described the 
static you hear when a modem connects as a hyper-accelerated Morse 
Code, a billion dots and dashes sung each second, too fast for the 
human ear to discern. This has been true since the dawn of networked 
computing. When the first two nodes of the Internet, at UCLA and 
Stamford, were brought online in 1969, Charlie Kline at UCLA famously 
initiated the connection by typing "login." After keying the letter 
"l" he received the appropriate echo back along the phone line from 
Stamford. The same with the letter "o." But when he hit "g" the 
system crashed; the audible reply from Stamford never reached its 
destination.

In 1972, Ray Tomlinson modified a program meant for ARPANET, the 
precursor to the Internet, that would let people send each other data 
as small "letters." He chose the  {AT}  sign for addresses for a simple 
reason: the punctuation keys on his Model 33 Teletype made it easy to 
type; it was a convenient way to lend a geographic metaphor to an 
otherwise abstract place made up of data and people's interaction 
with the nodes that hold the data together. In one fell swoop, 
Tomlinson signaled that data could be both a place and a linguistic 
placeholder for digital information as a complete environment. By 
using the  {AT}  symbol, he restated what modernist artists and composers 
had been pointing out for over a century: when information becomes 
total media in the Wagnerian and the Nietzschian sense in, we arrive 
at the "Gesamkunstwerk" or "the total artwork." The Situationists 
referred to this as a "psycho-geography." Antonin Artaud wrote an 
essay about it called "Theater and It's Shadow;" for him it was based 
on the interaction of different forms of alchemy. When Artaud coined 
the term "virtual reality" in his 1938 essay "The Alchemical 
Theater," he anticipated a realm where signs, symbols, letters, and 
ciphers were all placeholders in the rapidly changing landscape of a 
society that faced the surging tides of industrial culture's mad race 
to become an information culture. It was a phrase to describe a mind 
trying to make sense of the data road kill on the side of the 
information highway being built in the minds of artists whose dreams 
punctuated an immense run on sentence typed across the face of the 
planet as technology carried the codes out of their minds and into 
the world. In the 20th century, one symbol -- " {AT} " -- ushered in a new 
world linked by the intent of people to communicate. This is a world 
of infinitely reflecting fragments, vibrating, manifesting a hum, 
making music.

The connection between sound and networked computing is more than the 
product of technical convenience. It can be traced to the first 
visionary articulation of the digital age. In his seminal essay from 
1945, "As We May Think," Roosevelt's science advisor, Vannevar Bush, 
proposed the creation of a device he called the memex, which provided 
the inspiration for what later became the networked personal 
computer. Bush's memex system had the ability to synthesize speech 
from text, and, conversely, to automatically create text records from 
spoken commands. He wrote enthusiastically of the Voder, which was 
introduced at the 1939 World's Fair as "the machine that talks." "A 
girl stroked its keys and it emitted recognizable speech," Bush 
wrote. "No human vocal cords entered in the procedure at any point; 
the keys simply combined some electrically produced vibrations and 
passed these on to a loud-speaker." Bush also discussed another Bell 
Labs invention, the Vocoder, an early attempt at a voice recognition 
system. Central to his vision of the memex was the notion that sound 
would circulate through the system, available for easy retrieval and 
manipulation.

Today that ease of access and malleability is transforming the way 
musicians conceive of and make music. It is now simple to convert 
sound into digital streams, so it can flow anywhere across the 
computer network, to be manipulated by a continually growing array of 
software. Real time collaborations between musicians across the Net 
are becoming common. Online collaborations that are not real time are 
commonplace. The combination of databases (for storage), software 
(for manipulation), and networks (for interactivity between 
databases, software, and musicians) is challenging many long held 
notions of what music making can or should be. Established boundaries 
are blurring.

This blurring comes from a basic premise behind computing: that all 
information can be translated from its original form into binary 
code, and then re-articulated in a new form in a different medium. 
Texts can be stored in a database as ones and zeros, and later output 
as images or sounds. Ted Nelson, the man who coined the terms 
"hypertext" and "hypermedia" in the mid-1960s, was among the first to 
appreciate the full range of opportunities that networked computers 
make possible. In 1974, he proposed the playful idea of 
"teledildonics," a computer system that would convert audio 
information into tactile sensations. Why should music only enter the 
body through the ear? Why not through the skin, or through the eye?

Artists have been using computer networks for collaboration at least 
since 1979, when I.P. Sharp Associates made their timesharing system 
available to an artist's project called "Interplay." Organizer Bill 
Bartlett contacted artists in cities around the world where IPSA 
offices were located, and invited them to participate in an online 
conference -- essentially a "live chat" -- on the subject of 
networking. At the time this technology was rare and expensive; 
artists had no access to it. "Interplay" is often referred to as the 
first live, network-based, collaborative art project.

Around the same time, the innovative use of satellites by artists 
such as Nam June Paik, Joseph Beuys, Douglas Davis, Kit Galloway, and 
Sherrie Rabinowitz were connecting performers across great distances 
in collaborative, interactive pieces. A dancer in New York would 
improvise to music played in Paris, while video of the two would be 
edited into a single performance for broadcast in, say, Berlin. 
Although these pioneering telematic works did not make use of 
networked computing -- bandwidth and processor speeds were not yet 
great enough to allow for it -- they set precedents for the real time 
network-based interaction between artists that became possible in the 
1990s, as the technology improved and costs came down.   

Online collaboration today takes many forms. Using Web-based music 
technologies, artists are working together to create new music. There 
are online studios that connect artists across great distances, and 
Web-based jams between musicians who have never laid eyes on one 
another. At the same time, even more popular are "collaborations" 
between artists who are not even aware that a "collaboration" is 
taking place. Referred to as "remixes" or "bootlegs," digital files 
of a wide range of recorded material are being cut up and manipulated 
into entirely new works of art -- blending distinct and unlikely 
source materials into singular creations. Of course, this kind of 
unsolicited collaboration challenges some long-held notions of 
intellectual property, and an artist's unique affiliation with his or 
her own output. But at the same time, it brings back the idea of a 
shared folk culture, where creative expression is the property of the 
community at large, and can be shared for everyone's benefit. Digital 
technology may be a route that reconnects us to aspects of our tribal 
roots.

As new as these techniques are, however, they retain a continuity 
with pre-digital compositional approaches. The network simply allows 
musicians to perform together online, replicating the experience they 
have always had when jamming in the same room. At the same time, the 
mixing of distinct aural elements certainly does not require digital 
technology; analog sound mixing dates at least to John Cage's 1939 
performance of Imaginary Landscapes, which featured a mix of 
turntables and radios. From this perspective, computer networks 
simply contribute to long standing tendencies in composition that 
preceded the digital era.

However, some composers are exploring a wholly original, uncharted 
musical terrain, one that is unthinkable without networked computers. 
In these works, the sound experience is created through the real time 
participation of the listener in the making of the performance 
itself. These online sound art pieces rely on the interactive 
engagement of the listener, who helps to shape the specifics of the 
performance through her choices and actions, which are communicated 
to the music making software over the wired network. In this way, the 
traditional distinction between "artist" and "audience" begins to 
melt away, as the "listener" also becomes a "performer."


2. Composing With Software

When the software conditions the experience, it conditions the music. 
One thing that many people notice when they start making music online 
is that the Web is a powerful vortex; it doesn't let you go. There is 
no single way to end a session; rather, there are many ways. There 
are bootlegs of everything that has ever made it onto the Net. 
Multiplicity is an unwavering factor in the online experience. Look 
at sites like Afternapster.com. You will find hundreds of 
peer-to-peer networks, each of which is the private preserve of a 
file sharing community. These can be seen as the operational mode of 
a culture of distributed networks, held together by a common thread: 
each represents a particular taste as distributed through the system.

As Artaud said (in an incredible pre-cognition of the digital era's 
constant stream of information guiding any creative act): "All true 
alchemists know that the alchemical symbol is a mirage as the theater 
is a mirage... [It's] the expression of an identity existing between 
the world in which the characters, objects, images, and in a general 
way all that constitutes the virtual reality of the theater 
develops..." In a way, collaborative music making on the Net requires 
an interaction of people and software that turns almost all normal 
contact between musicians into a mediated experiment with the 
hypothetical. Is there a human on the other side of the screen? The 
sounds can only give you a hint. The software is a window onto a 
realm governed only by the uncertainty of that fact. The connections 
displace physicality in a way that leaves you a victim of context. 
This is the experience of tele-composing. It makes the creative act 
become a cog in the abstract machines of the software that mediate it.

Using online studio software, such as Rocket, Pro Tools, or Reason, 
allows you to mix equally with either musicians or found sounds. 
Through the software interface, there is a certain equivalency. 
Collaboration can take place in real time between people, or between 
the remnants of creativity that people leave behind -- the Net is 
full of such suggestive debris. In this context, the only limitation 
comes from the bottleneck that bandwidth places on file exchanges. 
The quicker the speed, the richer the environment.

Another effect of software is to dematerialize the musical 
instrument. It does this by distributing the qualities of an 
instrument across the various peripherals that control the sounds 
that the software generates. Algorithm displaces rhythm and becomes 
the  environment in which to create. MAX/MSP is an open ended 
software environment that lets you create templates for virtual 
instruments -- it allows you to make an aggregate of whatever sounds 
you run through its parameters. Almost all process oriented software 
behaves like this. Editing environments such as Pro Tools or Digital 
Performer function as dissecting tables of sound; they allow the 
musician to compose material from multiple layers of tracks and 
files, and to then condition the total output. It's like building 
music out of Lego blocks.

That is, either Lego blocks or samples. Online, everything is a 
sample. Every audio element becomes a potential fragment for 
manipulation and recontextualization. Sampling follows the logic of 
the abstract machinery of a culture where there are no bodies - just 
simulations of bodies. The fragment speaks for the whole; the whole 
is only a single track drifting through a vast database. The basic 
structure of "assemblage," the method of collage, holds sway here. 
 Think of this terrain as object-oriented programming with beats. 
Take the file, edit it: import/export/MIDI/SMTP.

Time code synchronizes the fragments, and makes it work wherever you 
are... FTP controls the data exchange as a basic source of file 
exchange... Lee Perry popularized the term "versioning" by using a 
series of vocal tracks that were taken out of context and 
de-familiarized through sound effects programming. This can be done 
either as a live process or improvised on a virtual "mixing board." 
Software that allows real time online jamming is appearing from every 
corner of the globe. But is your online collaborator a person or a 
bot? Or a combination of the two?  

On the Web, collaborators can come in all guises. The White Stripes 
have no bass player. Steve McDonald, the bass player for Red Kross, 
felt that the White Stripes tunes could use more bottom. So each week 
he adds a bass guitar part to one or two White Stripes songs, and 
makes them available as "bootleg" MP3s. Jack White, the White 
Stripes' front man, has apparently given these remixes his blessing.


3. Interacting With Intelligent Networks

Once, every sound had a distinct source. A door slammed shut, a horn 
was blown, a guitar string was strummed. Audio came from a discrete 
event, it was tied to a discernable action.

Networked music challenges this notion by displacing sound from its 
origin, moving audio freely from one location to another, giving it a 
presence in and of itself. John Cage brought this quality into modern 
music with his 1939 piece, Imaginary Landscape. A performance that 
combined turntables and radio broadcasts, this work introduced 
networked interactivity into music making.. Cage mixed into his 
performance various transmissions that came over the airwaves, and 
with them created an entirely new composition. Sound separated from 
its source in this manner becomes a "free floating signifier," to 
borrow a phrase from Roland Barthes. The musical elements are 
liberated from a specific time and place, allowing them to be 
recontextualized in the final composition.  

Robert Rauschenberg pursued something similar in the mid-1960s with 
his interactive, sound-emitting sculpture, Oracle. Rauschenberg's 
collaborator on the project, Bell Labs engineer Billy Kluver, 
described Oracle as "a sound environment made up of five AM radios, 
where the sounds from each radio emanates from one of the five 
sculptures. The viewer can play the sculpture as an orchestra from 
the controls on one of the pieces, by varying the volume and the rate 
of scanning through the frequency band. But  they can not stop the 
scanning at any given station. The impression was that of walking 
down the Lower East Side on a summer evening and hearing the radios 
from open windows of the apartment buildings."

By the early 1970s, as the technology became more accessible, more 
artists began to explore the potential of networked media -- both 
audio and video -- to create unique forms of interactive expression. 
These artworks grew from the notion that meaning would emerge from 
media as it circulates freely within a network -- and that meaning 
can be enhanced through strategic interventions by the artist or 
audience. Douglas Davis' 1971 performance, Electronic Hokkadim, 
produced at the Corcoran Gallery, was based on the interactions 
between telephone callers and broadcast television. Nam June Paik 
pursued what he referred to as a "cybernated art," based on the 
transmission of information through video and audio networks. Paik's 
1973 television broadcast, Global Groove, stands as a landmark event 
in this trajectory. Fragments of performances by artists of various 
traditions -- Western and Eastern, popular and elitist, traditional 
and modern -- were strung together in a frenetic, continuous flow 
across the screen. Paik himself "performed" the broadcast as a live 
mix, choosing his streams as a DJ does today, manipulating images 
through a video synthesizer, using rhythm as the underlying principle 
of composition.   

Enabling and manipulating the continuous flow of information was a 
principal concern behind the design of the networked personal 
computer. But before the mid-1980s, bandwidth constraints and limited 
processing power made the use of these tools prohibitively expensive 
for artists. However, it was long apparent to the pioneers of 
networked media -- such as Davis, Paik, and Roy Ascott -- that their 
artistic explorations with satellites and local wired networks would 
lead to computer-based work, once the technology had caught up to 
their vision.

Among the earliest musicians to dedicate themselves to the potential 
of networked computing were The Hub, perhaps the world's first 
"computer network band," which was founded at Mills College in 1985. 
One of the members describes their method as follows: "Six individual 
composer/performers connect separate computer-controlled music 
synthesizers into a network. Individual composers design pieces for 
the network, in most cases just specifying the nature of the data 
which is to be exchanged between players in the piece, but leaving 
implementation details to the individual players, and leaving the 
actual sequence of music to the emergent behavior of the network. 
Each player writes a computer program which make musical decisions in 
keeping with the character of the piece, in response to messages from 
the other computers in the network and control actions of the player 
himself. The result is a kind of enhanced improvisation, wherein 
players and computers share the responsibility for the music's 
evolution, with no one able to determine the exact outcome, but 
everyone having influence in setting the direction. The Javanese 
think of their gamelan orchestras as being one musical instrument 
with many parts; this is probably also a good way to think of The Hub 
ensemble, with all its many computers and synthesizers interconnected 
to form one complex musical instrument. In essence, each piece is a 
reconfiguration of this network into a new instrument."

Implicit in this approach is the idea that, within the network, a 
kind of intelligence is in circulation. David Wessel, at the 
University of California at Berkeley, has been working with his 
colleagues along these lines since the late 1980s, bringing together 
the fields of computer music and neural networks. Could an instrument 
 become intelligent, and adapt to in an automated manner to a 
musician's playing style? Could it learn the preferences of a 
particular musician, and modify itself in response to what it learns? 
Using the MAX programming environment, Wessel began to experiment 
with musicians in a network context. "We have obtained reliable 
recognition of complex guitar strumming gestures and limited numbers 
of spatial gestures," he wrote. "With such procedures and much more 
research, we might conceivably move towards adaptive, personalizable 
instruments.... one will have to decide when to standardize or fix 
the instrument and let the musician learn the appropriate gesture and 
when to let the instrument adapt to the specialized approach of a 
player. How to rig the training harnesses on ourselves as players and 
on our instruments as expressively responsive musical tools will be a 
question of scientific, aesthetic, and social concern." Once 
meaningful information is circulating within a computer network, the 
opportunity emerges for a relevant interaction. As Wessel suggests, 
networked computer tools will lead musicians into making choices 
about aspects of their performance that had previously never had to 
be asked, such as: how "smart" do I want my instrument to be?

The notion that music can emerge from an intelligent, interactive 
environment has drawn some composers to compositional forms that 
would be inconceivable without telecommunications technology. One 
example is Atau Tanaka's 1998 installation, Global String. The work 
consists of a physical string, 15 meters long, that stretches from a 
floor diagonally to the ceiling of a room. At the ceiling, the string 
is connected to the Internet. "It is a musical instrument wherein the 
network is the resonating body of the instrument through the use of a 
real-time sound-synthesis server," writes Tanaka. "The concept is to 
create a musical string (like the string of a guitar or violin) that 
spans the world. Its resonance circles the globe, allowing musical 
communication and collaboration among the people at each connected 
site.."

Ping, a site-specific sound installation by Chris Chafe and Greg 
Niemeyer, takes a similar approach. Ping has been described as "a 
sonic adaptation of a network tool commonly used for timing data 
transmission over the Internet. As installed in the outdoor atrium of 
SFMOMA," for the millennial exhibition 010101, "Ping functions as a 
sonar-like detector whose echoes sound out the paths traversed by 
data flowing on the Internet. At any given moment, several sites are 
concurrently active, and the tones that are heard in Ping make 
audible the time lag that occurs while moving information from one 
site to another between networked computers." In effect, Ping makes 
music out of the data flow of the Net -- the constant motion of 
digitized fragments in real time is given an aesthetic form.

The composer and theorist Randall Packer has explored this line of 
telematic composition in a number of pioneering collaborative 
installations. For Mori, an "Internet based earthwork" first mounted 
in 1999 by Packer with Ken Goldberg, Wojciech Matusik, and Gregory 
Kuhn, the trembling movements of California's Hayward Fault are 
picked up by a seismograph, converted into digital signals, and sent 
over the Internet to the installation. This data stream triggers a 
series of low frequency sounds that vibrate through the installation, 
viscerally connecting the visitor to the moment-by-moment 
fluctuations of the earth's actual movement.

In what he has referred to as "artistic research projects," Packer 
has further explored the possibilities of interactive, telematic 
musical works. One such installation, Telemusic, was staged by Packer 
and his collaborators Steve Bradley and John P. Young at the Sonic 
Circuits VIII International Festival of Electronic Music and Arts in 
St. Paul, Minnesota, in November, 2000. Telemusic brought together 
live performers, audio processing of their performances, and real 
time participation from the public through a Web site, 
www.telemusic.org. As the performers read from a script, their 
delivery was effected by audio processing triggered by the mouse 
clicks of visitors to the Web site. The final mix in the room was 
then streamed to the Web site, so a visitor could hear the final 
musical composition that she had contributed to by clicking a mouse. 
In order to create this direct form of interactivity, Packer's team 
had to develop an interface between impulses captured over the 
Internet and a server hosting MAX software. This circular experience, 
in which listener is also a participant in the making of a musical 
work, is indicative of the direction where the Internet is suggesting 
that music should go -- as the distinction between "artist" and 
"audience" begins to slip away, and we find ourselves dipping into 
the data flow, listening to the music that it makes, and that we make 
with it.



============================================================================
"None are more hopelessly enslaved than those who falsely believe 
they are free...."
Johann Wolfgang von Goethe


Port:status>OPEN
wildstyle access: www.djspooky.com

Paul D. Miller a.k.a. Dj Spooky that Subliminal Kid

Office Mailing Address:

Subliminal Kid Inc.
101 W. 23rd St. #2463
New York, NY 10011
#  distributed via <nettime>: no commercial use without permission
#  <nettime> is a moderated mailing list for net criticism,
#  collaborative text filtering and cultural politics of the nets
#  more info: majordomo {AT} bbs.thing.net and "info nettime-l" in the msg body
#  archive: http://www.nettime.org contact: nettime {AT} bbs.thing.net