How A.I. Is Quietly Revolutionizing Animation and Visual Effects in Film Industry

How A.I. is Revolutionizing Animation and VFX

A.I. Assisting Animation and VFX Crews for Years

While debates about the use of A.I. take center stage across the entertainment industry, the technology has been quietly assisting animation and visual effects crews for years. It has made some of the most astonishing visual images possible when artisans have been asked to do what was previously thought impossible.

Using A.I. for Specific Problems

When helmer Peter Sohn wanted characters based on the elements of fire, water, air, and earth for his new film “Elemental,” VFX supervisor Sanjay Bakshi and his team at Pixar looked to A.I. to make the process smoother. The look of the characters depended on adjustments that would align them with Sohn’s vision.

Neural Style Transfer

“We used A.I. for a very specific kind of problem, and we used a machine learning algorithm called neural style transfer,” says Bakshi. “Our animation is so highly scrutinized. We go through so many review cycles for every shot, the animators are handcrafting it, and there are not a lot of places where machine learning is applicable in the current form.

Using A.I. for Flame Stylization

“But on ‘Elemental,’ we have this one problem where we run these fire simulations on top of the characters to make them feel fiery. Then the flames themselves go through a pyro simulation that is very realistic. It’s a fluid simulation, a real temperature simulation. So, the flames that it produces are very realistic. We needed a way to stylize those flames themselves. As you can imagine, stylizing a simulation isn’t an easy problem. It’s just so temporal. It’s constantly changing. And that’s the beauty of fire. It’s always so different, so it’s mesmerizing to look at. So there are not a lot of techniques out there to stylize flames, but we found one, which is called neural style transfer, and that’s the technique we used. It was the only tractable solution.”

A.I. as a Technology with More Uses

Gavin Kelly, a founding partner at the Dublin-based Piranha Bar, an animation and VFX house, also sees A.I. as a technology that will come to have more uses as animators and content creators look to push the limits of their visuals.

A.I. for Motion Capture

“At the far end, and we’re not quite there yet, you just film something and then just tell A.I. what you want to change it into in terms of performance capture,” says Kelly. “So, with performance capture, it’s very complex. You’re putting the animation thing together, getting a face headset in place, and talking to the software that will talk to the hands, the body, and everything. Those are all different bits, getting everything to talk together. To create this pipeline, it’s very, very complicated. And there’s a lot of troubleshooting along the way. So, there’s no doubt that there were A.I. movement capture solutions in the past. We’ve looked at them before. They’ve been awful and not production-ready. We are now very close to production-ready to roll the camera, and A.I. will work it out and be robust. And it won’t shake, and it will look very convincing.”

It still Requires Adjustments from Artists and VFX Crews

For Bakshi and his team, A.I. still requires careful adjustments from artists and VFX crews to get the visuals where they want them to go. Nothing can be taken for granted.

“The person who worked with us on A.I. was Jonathan Hoffman, and he described it like throwing fish into a tornado and hoping to get sushi out of these machine-learning algorithms,” laughs Bakshi. “So you can input what you want, and you may get something really beautiful, but it still might not be what you wanted to get from the animation that comes back to you.”

The post How A.I. Is Quietly Revolutionizing Animation and Visual Effects in Film Industry appeared first on Asume Tech.

from Technology - Asume Tech https://asumetech.com/how-a-i-is-quietly-revolutionizing-animation-and-visual-effects-in-film-industry/