Are SecondLife Avatars, 3D Animations and Motion Capture Ready for Dancers?
Summary: There are a number of efforts underway to make the 3D animated human form more lifelike. These developments are taking place both in virtual worlds such as SecondLife and with 3D animations initiated through motion capture systems. The end result will be the creation of personally-identifiable animations that move and act as their real-world counterparts.
These advances in 3D animation coupled with improved and less expensive capture technologies and animation software will, I believe, lead to large numbers of dancers experimenting with virtual worlds and different approaches to creating more realistic animations. Quality dance animations offer new avenues for creativity, marketing, studying choreography, revenue generation and the re-staging historical dance works. But the question remains how long it will take before the tools and software will be realistically ready for the dance community.
In this post, I cover:
* The Avatar Puppeteering project from Second Life
* Hands Free 3D's movement-based approach for controlling avatars
* The Laban Motion Capture Project at NYU
* The PhaseSpace active marker optical motional capture system
* Facial capture from Mova
Personalizing Behaviors of SecondLife Avatars
A limitation with 3D virtual worlds such as Second Life is that it is difficult for users to express personalized behaviors, expressions and movements through their on-screen avatars.
This year Linden Lab, creator of Second Life, has re-launched an effort called Avatar Puppeteering that is intended to make avatars more expressive.
A key component of this puppeteering initiative is to create "physical" avatars with human-like joints that can be manipulated by being "pushed, pulled, or rotated in real time for maximum expressivity and responsiveness." And these real-time adjustments in movements can be made while not allowing the avatar to rotate in ways that are unnatural to humans. You can't, for example, rotate your avatar's hips 180 degrees while both feet remain facing forward.
The following two videos from the Avatar Puppeteering project show how the joints of these avatars can be manipulated:
Controlling Avatars with 3D Video Cameras
An additional approach to simplifying the real-time, personalized manipulation of avatars is being developed by Hands Free 3D. I've highlighted their first video before that shows how a 3D video camera can be used to capture human movement that controls a Second Life avatar.
(Their second video shows how a 3D video camera can be used to control objects in SecondLife by simply using gestures.)
Laban Capture Project at NYU
While Avatar Puppeteering may add more personal characteristics to avatars and Hands Free 3D may make it easier to manipulate on-screen navigation and objects, the on-going Laban Project at New York University is addressing the high-end of the spectrum.
Christoph Bregler, associate professor of computer science, and Peggy Hackney, director of IMS Berkeley Laban/Bartenieff Certificate Program, are combining their respective expertise in motion capture and Laban Movement Analysis (LMA) to create personally-identifiable 3D animated models.
The current state of motion capture does not allow viewers of the captured animation to identify the unique "movement signature" (an LMA term) of the actor whose movements are being played back. You're simply watching an impersonal skeleton or animated character moving in 3D space. The resulting animated character can be designed to take just about any shape or form and be infused with its own personality. The point of the NYU Laban Capture project is, however, to directly capture the unique movement qualities of a live subject through a motion capture system, which is a difficult undertaking.
For background purposes, here's a good video of how the motion capture process works. In this instance an optical motion capture system with active markers from PhaseSpace is being used in conjunction with MotionBuilder and Maya software from Autodesk.
And here's the end-result of such a capture process, also from PhaseSpace, of a rhinoceros Flamenco dancer:
Mocap Already Available for Expressive Faces
When it comes to capturing and animating unique facial expressions lots of progress has already been made from a company called Mova.
This first video shows the animated results of the face-capture process:
This video shows how the facial expressions were captured:
Your Feedback and Thoughts
Are you thinking of using motion capture or performing in SecondLife? Please share your thoughts and stories.
Posted by
Doug Fox on July 8, 2008 5:20 AM
Trackback Pings
TrackBack URL for this entry:
http://greatdance.com/mtadmin/mt-tb.cgi/2759
However, as always and all too often, it is clear that a limited population of movers have been used in these examples. When you want to capture "authentic" movement to transfer to the virtual world, you have to have a variety of movers in order to create more genuine movement patterns. This variety needs to include people of various ethnic backgrounds because cultures utilize 3-dimensional movement differently. Each of us is a product of our environments - and lack thereof -and REAL people pick up the movements of others (on subconscious levels) in their environments .
When you deal with a limited population that predominantly utilizes vertical and horizontal planal movement, the richness created by full rotation from "adventures into the sagittal realm" are lost. We see this in how dancers who study only one form of dance (Ballet, for example, without the benefits of training in Jazz or any form of ethnic dance that provides for more 3D-free-form movement) move in general - when not dancing. We see this in how people walk down streets in big cities (where a variety of cultures meet) compared to small towns (where everyone is of the same or similar cultures).
In order to create an artificial entity's movements to be more realistic in general, you will have to move past the limitations of the the REAL people you know and have used for these projects and combine the movements of new people from different cultural and ethnic backgrounds to create a more realistic "Earthling" in cyberspace.
"it is clear that a limited population of movers have been used in these examples."
From a CMA (Certified Movement Analyst - an expert in Laban Movement Analysis) perspective, how do you see above video of rhinoceros as being reflective of the other movement styles represented in the above videos?
I don't know enough about the range (or lack of range) of different movement styles that are typically used in the motion capture process. But I think that you raise a very good point about making sure that when capturing, analyzing or animating movement styles that you don't limit yourself to only a limited number of different movement styles.
But then what are the sources of different movement styles? Is it possible to say that people from different ethnic backgrounds tend to have specific movement patterns and styles and what are they? Or are we just creating generalizations that don't work on the individual level?
I'm sure a lot of research has been done in this area - I look forward to learning more about it.
I am artistic diector of Ballet Pixelle (formerly Second Life Ballet), the first ballet company in virtual space (we have been performing in Second Life for going on 2 years). Our goal is to investigate the intersection and interaction of physical and virtual dance. To that end, it is nice to both be able to create animations in a number of programs, say Poser, and bring in classical dance vocabulary for ballet, as well to create movements that are impossible in real life (without joint limitations, for example). However, clearly a motion capture system of low cost and high quality would bring in a number of interesting qualities based on the dancer and an interesting kind of moves based on the choreographer.
~ Inarra Saarinen
artistic director
Ballet Pixelle
(formerly Second Life Ballet)
It would be great if you could expand a bit on the different software programs you use to create your SecondLife avatars and the motion sequences during performances.
How do you use Poser and the other software programs you use? What is the learning curve for other dancers who want to perform in SL and other virtual worlds?
I was really focused on human movements, not human movements imprinted on animals (mostly because I prefer to see real animals in motion, especially since more species are becoming endangered daily) because of the amazing advancements in video games, which still lack realistic, 3D, human movement styles and patterns.
A suggestion for a study is to have several dancers trained in different dance styles (e.g., Ballet, Jazz, African -- and various techniques of each) perform a series of simple movements - example, 4 steps forward, 360-degree revolution, move to place low, place high - and see how the movements are integrated throughout their bodies. Seriously focus on how breath supports movement and I'll bet that will give great guidance in developing realistic torso motion.
Another suggestion would be to take women of different cultural backgrounds and have them walk across the room wearing high heels. (My strong suggestion would be NOT to use runway models. I think that "runway walk" should be left to the Lipizzaner Stallions.)
From observation of the factors that make the differences in movement styles of people of different cultures and dance training, you will get a more vivid "map" of how to simulate real human motion.
There are issues that must be factored in, such as the fact that you can have several people from the same culture and dance training perform the same movements and see their "movement histories" present themselves with great variances in movement expression! But that part could just be the icing you could put on the cake, once you get the simulation to be closer to realistic, 3D human motion. I see this as being useful for developing "movement personalities" for the simulations.
I am fascinated by these types of projects because I foresee them being very useful in physical rehabilitation.
Best of luck and please put me on your mailing list to be kept up to date!
The Kinetic Interface blog on Great Dance starts with the premise that by focusing on the body and movement we can better understand, engage with, and contribute to many of the technological and scientific changes that are reshaping our daily lives.
To share stories and ideas, offer feedback, and ask questions, please email Doug Fox.
5 Comments
GREAT ideas!
However, as always and all too often, it is clear that a limited population of movers have been used in these examples. When you want to capture "authentic" movement to transfer to the virtual world, you have to have a variety of movers in order to create more genuine movement patterns. This variety needs to include people of various ethnic backgrounds because cultures utilize 3-dimensional movement differently. Each of us is a product of our environments - and lack thereof -and REAL people pick up the movements of others (on subconscious levels) in their environments .
When you deal with a limited population that predominantly utilizes vertical and horizontal planal movement, the richness created by full rotation from "adventures into the sagittal realm" are lost. We see this in how dancers who study only one form of dance (Ballet, for example, without the benefits of training in Jazz or any form of ethnic dance that provides for more 3D-free-form movement) move in general - when not dancing. We see this in how people walk down streets in big cities (where a variety of cultures meet) compared to small towns (where everyone is of the same or similar cultures).
In order to create an artificial entity's movements to be more realistic in general, you will have to move past the limitations of the the REAL people you know and have used for these projects and combine the movements of new people from different cultural and ethnic backgrounds to create a more realistic "Earthling" in cyberspace.
Good luck and much success!
Denise,
Thanks for comment. You raise good points that offer much food for thought.
For starters, I had to look-up "sagittal."
For my clarification: You wrote:
"it is clear that a limited population of movers have been used in these examples."
From a CMA (Certified Movement Analyst - an expert in Laban Movement Analysis) perspective, how do you see above video of rhinoceros as being reflective of the other movement styles represented in the above videos?
I don't know enough about the range (or lack of range) of different movement styles that are typically used in the motion capture process. But I think that you raise a very good point about making sure that when capturing, analyzing or animating movement styles that you don't limit yourself to only a limited number of different movement styles.
But then what are the sources of different movement styles? Is it possible to say that people from different ethnic backgrounds tend to have specific movement patterns and styles and what are they? Or are we just creating generalizations that don't work on the individual level?
I'm sure a lot of research has been done in this area - I look forward to learning more about it.
I am artistic diector of Ballet Pixelle (formerly Second Life Ballet), the first ballet company in virtual space (we have been performing in Second Life for going on 2 years). Our goal is to investigate the intersection and interaction of physical and virtual dance. To that end, it is nice to both be able to create animations in a number of programs, say Poser, and bring in classical dance vocabulary for ballet, as well to create movements that are impossible in real life (without joint limitations, for example). However, clearly a motion capture system of low cost and high quality would bring in a number of interesting qualities based on the dancer and an interesting kind of moves based on the choreographer.
~ Inarra Saarinen
artistic director
Ballet Pixelle
(formerly Second Life Ballet)
Inarra, thanks for your comment.
It would be great if you could expand a bit on the different software programs you use to create your SecondLife avatars and the motion sequences during performances.
How do you use Poser and the other software programs you use? What is the learning curve for other dancers who want to perform in SL and other virtual worlds?
Thanks!
Hi Doug,
I was really focused on human movements, not human movements imprinted on animals (mostly because I prefer to see real animals in motion, especially since more species are becoming endangered daily) because of the amazing advancements in video games, which still lack realistic, 3D, human movement styles and patterns.
A suggestion for a study is to have several dancers trained in different dance styles (e.g., Ballet, Jazz, African -- and various techniques of each) perform a series of simple movements - example, 4 steps forward, 360-degree revolution, move to place low, place high - and see how the movements are integrated throughout their bodies. Seriously focus on how breath supports movement and I'll bet that will give great guidance in developing realistic torso motion.
Another suggestion would be to take women of different cultural backgrounds and have them walk across the room wearing high heels. (My strong suggestion would be NOT to use runway models. I think that "runway walk" should be left to the Lipizzaner Stallions.)
From observation of the factors that make the differences in movement styles of people of different cultures and dance training, you will get a more vivid "map" of how to simulate real human motion.
There are issues that must be factored in, such as the fact that you can have several people from the same culture and dance training perform the same movements and see their "movement histories" present themselves with great variances in movement expression! But that part could just be the icing you could put on the cake, once you get the simulation to be closer to realistic, 3D human motion. I see this as being useful for developing "movement personalities" for the simulations.
I am fascinated by these types of projects because I foresee them being very useful in physical rehabilitation.
Best of luck and please put me on your mailing list to be kept up to date!
D*
Leave a Comment