Backflip
Attempting a backflip is not safe. You can break your neck, or land on your head, or land badly on your wrists. None of that is nice, so the avatar does the trick. He can do 360 jumps per hour. The film is simulated based on the research paper «DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills» by Xue Bin Peng, Pieter Abbeel, Sergey Levine and Michiel van de Panne. The process allows to extract motion data by analyzing real-life videos of human specific tasks. The data is used as a reference to train a digital character to replicate a chosen task via Machine Learning. Specified learning stages are extracted and reused to simulate the character in a custom environment. By structuring these simulations one after another we are able to tell the story.For the film, the character learned a variation of tasks so that it can perform the majority of the action. The avatar and the world around it are constructed with the method of camera projection mapping. Realistically looking backgrounds and props reveal their true flawed and distorted shape during shifts of the camera angle. The camera is recorded in real time in VR following the character’s movements. The voice of the character uses text-to-speech (tts) and is the result of speaking 15min into a voice cloning software («descript»). This allows for unlimited text input. Lip syncronization is a combination of Wav2Lip (GAN), facial tracking and the application of the trackers to trigger facial joints. The music is automatically generated in aiva.ai and manually mastered. [Source]
«I find it very inspiring to observe people facing their biggest fears and challenges. That is why I like YouTube and its videos of progressions and tutorials on nearly every given task. One of the most prominent tasks is the backflip. If you search «how to backflip» or «backflip progression» on YouTube, you will find countless backflips by users from all age groups. Given that you are not a gymnast, it is difficult to imagine that you can perform such a miraculous trick. Precisely, you must stand on solid ground, jump in the air, flip 360 degrees backwards and land on your feet, which sounds scary if not impossible. However, as soon as you see that it can be done by people with similar physical body conditions as yours, it becomes a challenge, and the question is: Can I do it? I tried to do it in 2019. Back then, I was travelling, and decided to rent a place with multiple beds so that I could use enough mattresses and cushions to construct a safe landing ground. It still wasn’t safe, but there was no gym, and considering the setups on YouTube, this felt like the most honest and authentic way to learn. I was inspired to just do it and to my surprise, starting with the first clunky attempts and over the next 5 days of practice, I came very close. The biggest challenge during that period was to get rid of the fear. At the start, it was an obstacle, still, little by little and with every progress, the fear seemed to slowly fade and turn to confidence. Towards the end, I was spinning backwards and landing with my feet and arms on the mattress. I was optimistic. The next adjustment was to tuck my feet entirely, fully commit and land only on my feet. By that point, the rental period of the apartment was nearly over, I was tired, my body was sore, I was aching in places I never felt before and the whole setup was still unsafe. However, the proximity of succeeding and the rush of having to move out without a result motivated me to keep going. I should have stopped. The breaking point was my toe. I jumped too far towards the back and past the mattress so that my left big toe hit the concrete floor. The toe turned red, then blue. I tried to shake it off and thought to continue, but my mind shifted. I was paralyzed, the confidence was gone, and I couldn’t repeat any of the previous attempts. I was jumping upwards but couldn’t force myself to spin backwards. I felt the pain and was imagining that with every other jump, I might break something else. It was as if my mind tried to protect me and blocked the communication to the rest of the body. I was fully aware of it and that was the end. I have never tried a backflip ever since. I gave up and failed. Due to a broken toe. That is why I built my avatar. In a way, it is me coping with the above experience by virtually learning the backflip via a copy of myself. The avatar doesn’t feel any pain and would keep on going, no matter what. The avatar is learning with the help of Machine Learning. My friend Max who is a programmer, helped me to set up the training environment based on a paper called «Deep Mimic». The process is very similar to real life as the avatar observes videos of humans and then tries to imitate their movement. It also learns from its own mistakes and improves upon them, just like us. I find it both magical and scary. Magical because it resembles us so much, scary because A.I. technology seems to have no ceiling and it makes me feel anxious. I will not go into the doomsday views of Super Intelligence and Singularity; the field of A.I. is a well-documented complicated topic, and, in the end, I am not a scientist. All I know is that, as a bystander and filmmaker, it scares and fascinates me. Plus, it is infinitely thrilling to be able to play with something that is way above my head. All in all, «backflip» is about ambition. Sometimes blind ambition, both human and artificial. It is about fear or lack thereof. It is about control versus uncertainty, rationality versus emotion, and the desire to excel. It is about technology» – Directors statement by Nikita Diakur
