Group
For the collaboration project, Miranda and I were chosen to work with London temporary dance university students. The overall goal for our group is to animate the dancer’s dance in different forms, such as smoke, liquid, and feather. we will be filming dancers’ movements on the phone and also motion capture.
There are five people in our group:
- Jason Cui (BA Virtual Reality)
- Miranda (BA Virtual Reality)
- Algo AU (BA LCDS)
- Cynthia Cheung (BA LCDS)
- Melina (BA LCDS)
PROCESS AND COMMUNICATION
For the first time, we went to LCDS and met our group members to discuss ideas and record videos. Experiencing AI Rokoko Tool and Motion capture room gives us a deeper understanding and unique ideas for doing this. here are some videos that we use as a reference for our early stage.
My Initial thought is VR can create virtual environments that simulate unique performance spaces, from grand theatres to fantastical landscapes. Dancers can perform in these virtual spaces while wearing VR headsets, transporting themselves and their audiences to immersive worlds that enhance the overall experience.
by removing physical barriers, This enables dancers to explore new forms and shapes, and interactions beyond the limitations of the physical world. it makes it more tempting for me to create this inclusive vision.
Firstly we asked them to stand in a T-pose position, the reason T-pose is used to establish a neutral position with aligned joints. This helps ensure accurate tracking and mapping of movements in motion capture. By starting with the T-pose, dancers provide a consistent reference point for the system to effectively recognize and track their body movements.
For the task distribution, Miranda is doing real-time Animation which she will be using a projector where dancers will have to perform at the same time. I will be using mainly Blender to make different element effects for three of the dancers.
I’m glad that everyone seems to enjoy the process, for the LDCS students have to dance 5 hours a day, therefore I understand that it will take a long time for them to post the videos that we require, during this period of time Miranda and I are trying to book the motion capture room, the room is hard to book cos its usually used by animation student and we are lucky that have a chance to become the first two students from VR that experience the mocap suits.
Motion Capture & Green Room Recording
Miranda and I tried the mocap suit and showed it to the entire class, we also made some animation clips for our other project, it was an enjoyable experience, here is the video of the dancers dancing in the green room starting with the T_pose.
Continuous Green Room Recording
The purpose of the piece is to show the dancer’s style in a way that morphs from solid to fiery or smoky shapes that add a more stunning visual effect and attract the audience. To achieve this capturing effect I use mainly blender and cinema 4D
We aimed to portray the movement of each element through human interpretation of our project’s characters. To accomplish this, we allowed the dancers to choose an element based on their personalities and dance styles. for example, Cynthia Cheung is very introverted and has soft dance movements, which represent the water element, I think it will have a good combination together.
For the liquid and smoke effect, I used a video from Algo, and the body needs to be in a t-pose when the recording starts. then I use the Rokoko website to turn him into an animation, and then I found a character from Maximo as Algo’s body and import it into Blender with the animation.
Experience & Rendering issues
There are a few things that I did when importing a character from Mixamo make sure to get rid of the animation on the body and have the automatic bone orientation.
Then I use the Rokoko blender plugin to target the bone and body together which I found very helpful than adjusting the bones and clearing the parent original bones from the body by myself.
The experience that I have when making the liquid effect is incredible, by increasing the revolution divisions and changing the mesh and diffusion, it looks so realistic!
then I started to put lighting, texture and render the scene.
Updates
Unfortunately, certain videos recorded by the dancers cannot be utilized on Rokoko, as the computer is not capable to convert their dance moves into animations. Consequently, we requested them to reshoot the footage. However, as previously mentioned, it is time-consuming to await the submission of their new videos. at the same time, I used the animation from Mixamo and test it.
Also, we are shocked that the rendering is taking way too long, 2000 frames nearly took me a whole day to finish.. therefore After extensive discussions, we have concluded that it would be more advantageous to conduct the rendering process at LCC where these desktops possess greater processing power compared to our laptops.
Reference & Inspiration
I like this flame dude the whole body transforms from smoke to fire.
Sounds
https://www.youtube.com/channel/UCjuuXtlsal6UXVOyFdWJ9Dw