360 VR Compositing problem. (AE & Mettle Skybox)
I've come across a sticky problem whilst compositing some VR images that I've been banging my head against for days without success.
I have a series of 360 aerial stills (shot on a gopro omni) over which I need to animate map elements across the landscape. The map elements are vector graphics created in illustrator and traced over google map screen captures of the relevant area.
I order to appear in the correct position and properly track the landscape within skybox the illustrator maps often need to be scaled between 5000% - 20000% meaning I need to have constant rasterise switched on for the map layers in order to keep their sharpness... and this is where my problem begins.
The horizon in the 360 captures falls away in a different manner than that of 3d elements in after effects camera, so on particularly large maps the immediate foreground looks fine but as it gets further away from the camera the horizon falls away and the map elements look as if they're extending up into the sky. (See very rough image attached)
I need to be able to distort my map layer in order to make it fit the landscape. Now AE has any number of distort tools that would work with a normal pre comp, but as soon as constant rasterise is switched on I can get nothing at all to work, not even Mettle's own FreeformPro. The layer contents either disappear entirely or the distort effect continues to be 2d and not follow my 3d map layer. This applies regardless of those effects being applied to the 3d layer itself or inside the precomped 2d layer.
I've even tried giving up on the 3d composite altogether and tracing the map elements directly onto the equirectangular source image itself, but even with careful tweaking of stroke widths etc the perspective never quite looks right.
So I appear to be stuck between a rock and a hard place. I cannot accurately distort and manipulate my map layer without turning off constant rasterise and I can't turn off constant rasterise without dealing in pre-rendered layers in the region of millions of pixels to each side, which will obviously make AE spit it's dummy.
Deadlines are looming and progress is glacial. Can anybody suggest a solution to this predicament that I have not yet thought of?
Can you pre-compose the 3D tracked footage, and apply a fisheye-like distortion to the pre-comp to make it match more closely? Otherwise, you would have to apply some kind of distortion correction to your footage, and I don't know how well that would work with 360˚ footage.
What about using the depth of field option to add falloff to your objects in 3D space?
That does look like a rock and a hard place. I'm not sure how Mettle's tools work but if your map assets are 2D cards in 3D space then you should be able to angle them correctly for each 360 still. I noticed that your horizon line is not a straight line, are you doing this in each cube face and then outputting from there, or are you working in a 360 comp (in the attached still)?
There is another 360 toolkit for After Effects you can try, Canvas 360 Pro.
You can get the (free) Lite version here: http://torusmedialabs.com/
Canvas 360 Lite has limitations but if you're only using stills and a handful of layers in your composition then you should be fine (no watermark).
Here are some tutorials:
(scroll down and click on Canvas 360 Lite for the Lite tutorials)
Thanks for your considered response.
I'm working in 360 comps using 360 stills as the backdrop on to which I am animating various vector graphics, text etc and outputting to 360 VR video. The assets I am struggling with need to stretch smoothly from horizon to horizon so simply angling them is not a viable solution and working on individual cube faces leaves me with the same problem as painting on the original equirectangular images did ie; the perspective just does not look quite right when composited and viewed in a headset.
I did take a look at Canvas 360, but I didn't see any obvious way in which it might solve this particular issue and my company is already well down a development pipeline with Mettle and would be reluctant to completely reverse now.
After another long morning of headbutting a brick wall I finally gave up. I think this genuinely may be a case of the plugins having not yet caught up with the needs of the user in an emerging technology so I decided I needed to think outside of the after effects box and move to a software more suited to a full 3D environment. In my case Blender, but I think it would work in most 3D suites.
My work around was thus: Using my original vector graphics as a guide I recreated the necessary map elements in blender with bezier curves and used them to create meshes. Using the original 360 image for both reference and world lighting I then shrinkwraped these meshes to the top half of a sphere. Then, by manipulating the scale of this sphere in the Z(Y) axis I was able to roughly match the horizon fall off in the original image and get my map to match the landscape with reasonable accuracy and good perspective. I then rendered each individual map element as an equirectangular alpha which I can lay on top of my 360 image in AE and animate, tint, distort and generally faff with to my hearts content without worrying about rasterising, aliasing or anything else.
This is definitely a workaround and far from an ideal workflow (take note plugin writers) , but considering I'm working on a product that is due to go to market within a month I'm glad to have found any solution at all, hence why I reiterate it here for any future poor souls who run into similar issues.
I'm still very open to any suggestions that can keep my workflow entirely within AE so please continue to comment.
not sure why, but I only got notified by the COW about this post right now ☹ I will speak to admin at the COW to see if there is a glitch in their notifications (in some way).
In future feel free to reach out to me directly chris (at) mettle.com I would be happy to look at things closely with you.
One observation you made that is right on the money is that The AE architecture definitely has room for improvement, especially when it comes to 3D third party plugins. We are working very closely with Adobe and hope to have an even tighter integration with AE in the future.
In the interim, please check out our blog for some amazing customer success stories including some recent ones from Sundance 2017... we will post quite a few success stories from Sundance soon.
Lastly, I cannot state this often enough: #MettleCustomersRock ☺
Thanks for your patience and understand ing and feel free to reach out to me directly in the future if you are under the gun.