April 28, 2008

Will Muscle and Musical Memory Inform New Interface Designs?

Summary: Over many years of practice, dancers hone their muscle and rhythmic memories. They can watch extended dance sequences and often perform what they've just seen demonstrated after a single viewing...As new movement-based interfaces increasingly permeate the environments in which we live and work, will muscle and rhythmic memory become an important skill for successfully interacting with and responding to our digital surroundings? And what insights do dance-makers have about the challenges and possibilities of these human-computer interactions?

I've always been impressed by the muscle and rhythmic memories of dancers. When I started taking jazz and modern dance classes, my body simply could not remember a series of moves that were demonstrated by a teacher. But after about two years, my body seemed to develop a magical capacity to watch and then perform movement phrases. I'm still very much a beginner dancer, but it does feel good to be able to replicate sequences that I've just seen performed for the first time moments before.

In this video interview with choreographer Mark Morris, you can watch company members rehearse for "Mozart Dances," and get a glimpse of the highly developed rhythmic and muscle memory abilities of professional dancers:

I also encourage you to watch this related Mark Morris video, "Taking Sides."

When TV show host Ellen DeGeneres visited the dancers of Alvin Ailey American Dance Theater at their studio on 55th Street and 9th avenue here in Manhattan (a great place to take dance classes), she has fun with the talented dancers knowing that they can instantly watch and then "playback" her dance moves:

Last week for the fourth episode of the new reality TV dance show "Step It Up & Dance, the contestants had to quickly learn the challenging rhythmic patterns taught by the cast members of Stomp Out Loud:

Also, watch this clip from last week's Step It Up & Dance rehearsal with the Stomp crew as the dancers learn to do a trashcan lid routine.

Digital Interfaces Based on Muscle and Rhythmic Memory

What's interesting about muscle memory (remembering and playing back physical movements over time) and rhythmic memory (remembering and playing back sound patterns over time) is that they are both time-based. In other words, no value is derived by freezing the action or sound at a single point in time.

If you take a look at my video guide to digital interfaces, you'll notice that the movement of your body does control these digital tools and software applications. But in most cases there is only a one-to-one relationship between a specific movement and a response to that movement.

In this demonstration of a gesture-controlled robot, one movement of the user controls either the specific direction or rotation of the robot:

The demonstration of the "Shadow Reaching" technology is a bit more interesting. If you jump to the 47-second mark in the below video, you'll see how Shadow Reaching can be used to move objects on a large-screen display. You'll see that it takes three actions of the user (grab, drag and release) to complete the process of moving an object across the screen. I don't know the specifics of the software, but I'd imagine that each action "grab," "drag" and "release" is viewed as a discrete event by the program. But the overall effort of changing the position of an object does have the feel of a movement routine.

Questions for interaction designers and dance-makers:

- As digital interfaces become more complex and pervasive, will they begin to accept input in the form of movement and rhythmic patterns? (There may be examples out there already that I'd like to learn about).

- What types of interfaces might rely on this type of time-based input?

- How specifically would we control interfaces that used movement and rhythmic patterns?

- What insights can be learned about controlling these interfaces from dancers and others whose artistic work is based on the mastery of muscle and rhythmic memory? And,

- What are examples of dance works that have relied upon movement patterns to control interactive environments that are incorporated into a performance?

Trackback Pings