Choreographing Gesture Controls for Interactive Devices
Dan Saffer is the author of "Designing Gestural Interfaces: Touchscreens and Interactive Devices" (Author blog, Amazon). Published in December of last year by O'Reilly, this book addresses the increasingly important topic of how we should best design interactions for the many digital tools and technologies that rely upon human touch, the body and movement for input and control. The Apple iPhone and Nintendo's Wii are the most popular examples, and you can read extensive excerpts from Saffer's book on Google to learn about many other gesture-driven interfaces.
I'm especially interested in Saffer's book because it covers many topics that are directly related to dance, movement and choreography. There is a chapter on the basics of kinesiology, an overview of the leading dance notation systems, and an extensive picture gallery of poses that could be used to control gesture-based devices.
I would be very interested to learn what dance notators and Laban Movement Analysts think about developments in gesture-based interfaces in the context of their work and expertise. And I would be curious to know how choreographers would go about creating their own sets of movement vocabularies to control the many digital devices we come in contact with on a daily basis.
Dance Notation Systems
In Chapter 5 "Documenting Interactive Gestures," Saffer gives a brief introduction to Labanotation, Benesh Movement Notation and Eshkol Wachman. And he also writes about Laban Movement Analysis as a tool that can be used to help understand and analyze the dynamics and quality of different gestures. He explores these dance notation systems in his efforts to offer guidance to interaction designers on how they can document their work as they test different ways that our gestures can control computer interfaces.
After exploring the leading dance notation systems he writes:
All of the existing systems of movement notation are also notoriously painful to read and write.
And he believes that interaction designers will probably need to find "lighter weight alternatives" for their documentation purposes.
Most dance notation systems are difficult to learn and the online descriptions of these systems could be improved and expanded. But Saffer omitted coverage of Motif Description, a simplified sub-set of Labanotation, which might be an effective tool for documenting movement in the context of gesture-based interfaces.
To learn about Motif, you can visit the Labanotation Basics page on the Dance Notation Bureau site and click "Motif Description Basics." And the Language of Dance Centre publishes Motif at a Glance, a flip-chart dictionary of Motif notation symbols.
Gestures for Free-Form Systems
In the appendix, Saffer includes almost fifty pages of pictures of model Ellen Ho making a range of gestures and holding different poses. The purpose of this photo gallery is to propose a dictionary of static poses and dynamic movements that can be used to control devices in a specific way. For example, sitting, standing, twisting, bending and jumping can all have specific meanings in terms of controlling an interface.
--------------------------------------------------------
To see many of these images on Google Books, go to Appendix A for "Designing Gestural Interfaces" and scroll down to page 188.
As I mentioned above, I would be curious to see what choreographers would come up with if they created their own movement language to control devices that accepted free-form input.
Posted by
Doug Fox on March 25, 2009 6:15 AM
TrackBack URL for this entry:
http://greatdance.com/mtadmin/mt-tb.cgi/3085
2 Comments
Nice topic!
We had a Certified Movement Analyst (Jodi James) in our research group at Arts, Media and Engineering, and her knowledge was immensely useful to our research and interactive environment design. We did a lot of work in trying to capture LMA shape qualities (advancing/retreating, spreading/enclosing, rising/sinking) in general movement. You can see an interactive environment based on this analysis on blip.tv.
We never used Labanotation though... Storing and visualizing gestures was done almost exclusively via motion capture data or video. At times, it was also useful to visualize quantitative properties of gestures, such as how much deviation there tends to be at various points of a gesture (computed from multiple recordings of a gesture).
We are also developing a collection of libraries (AMELiA) including one for recognition of specific gestures / patterns. If anyone is interested in using it, there is contact info on the website, and I'm always more than happy to get people started with it.
Stjepan, thanks for sharing information about the Arts, Media and Engineering research group at Arizona State University. I also took a look at your related "View of the Istage" video.
I'm going to go back to your links above to learn about your gesture and pattern recognition research. I'm curious how that was done.
Getting back to my post, I think one of the main questions is what are the main benefits of each approach to documentation of gestures for interactive devices: video, mocap and dance notation systems?
Leave a Comment