ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

Tracking a 360 footage - Oculus Rift

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>

simone nunziato
Tracking a 360 footage - Oculus Rift
on Jan 4, 2014 at 12:19:28 am

Hello everyone,
Want to thank you first of all for keeping this forum everyday more interesting and useful

hope you can help me. I basically need to track a 360 footage and overlay some graphics on top of it, the footage is gonna be played in Oculus Rift.
Here you can see a screenshot:
Video is shot with 5 cameras and then stitched together.

Problem is that if I normally track the video and add the graphics on top they won't be distorted as if it was shot with a 360 camera.
Tried polar coordinates in AFX but it doesn't really help at all, also displacement map didn't really work.

Wanted to know if there's anything that can help to distort the graphics as they were embedded in the footage.

Return to posts index

Dave LaRonde
Re: Tracking a 360 footage - Oculus Rift
on Jan 7, 2014 at 4:59:04 pm

You might try asking about this on the Oculus Rift forum:

You may find there's a better method than using AE for this chore.

Dave LaRonde
Promotion Producer
KGAN (CBS) & KFXA (Fox) Cedar Rapids, IA

Return to posts index

Philip Bowser
Re: Tracking a 360 footage - Oculus Rift
on Jan 10, 2014 at 8:57:36 pm

This is a great question! And not one exclusive to the Oculus Rift. I had previously pondered about how one would go about match moving panoramic footage for an interactive short film I was going to develop. In all of my research it doesn't seem as though there are any tools built specifically for tracking this type of footage.

There were two potential solutions I came up with that I never put into practice, but here they are anyways:

Solution 1:
Track all cameras individually/unstitched in a tracking software (I use PFTrack). You can use the first camera track's position/rotation data as a "hint" for the second camera track, and then use the second camera's data as a hint for the third and so forth. You're going to want to align each of the cameras to each other in virtual space as they were set up in real world space. There are tools for this in PF Track but I'm not sure what software you would use. You then export all of your cameras to your 3D app or AE. You animate whatever magic you're trying to have happen and render it out separately from each 3D camera view. Then you stitch each of those renders back together using the same technique that you stitched the original footage together and if all works out perfectly for some reason it should line up! And Composite away!

Solution 2:
If your individual graphic elements will only be localized to a certain area in your panoramic footage, you may be able to get away with rendering out those portions of your panoramic video as regular HD footage (essentially cropping in to the 16:9 dimension you desire) and then do a camera track on just that portion, add your effects, render out, and then bring that render with your effects back into the panoramic composition and line it up to that original portion you rendered out.

Because this is such an unusual request, I doubt there are any easy tools out there for this kind of problem. And without any camera data or scene surveying recorded on set to make the tracking easier, it could prove to be extremely challenging to get anything usable. However if you find a solution please post about it, I would love to know how you end up solving this!!


Philip. Bowser

Return to posts index

clyde desouza
Re: Tracking a 360 footage - Oculus Rift
on Jul 20, 2014 at 1:05:52 pm

Interesting question for sure.
The clue must lie in the Lens distortion map of the camera. Maybe shooting a distortion grid and then aligning the Graphic element to this grid could be the workflow?

Check this post for a possible clue:

Please let us know how this goes.

Clyde DeSouza
twitter: @cly3d
Author: THINK in 3D: Food For Thought for Directors and Cinematographers.

now available on Amazon:

Return to posts index

Dylan Saliba
Re: Tracking a 360 footage - Oculus Rift
on Aug 31, 2015 at 3:13:32 pm

I just wanted to say I've been using the cow for YEARS as a rock solid reference to any A/V limitations / issues and usually the answer is found here! This is my first post, but it's a good one.
The solution I have found to this problem you list is a small script for after effects from Mettle called SkyBox. It basically sets up a 360 rig with AE cameras and then precomps to a nice and reasonably simple setup. The demo allows for 3 "skyboxs" to be created so you can totally check it out and see if it works for you before purchasing. It's only $65 too, completely worth every penny! With this script / plugin you can also have interactive HDRi plates...but that is a whole other can of worms. This 360 stuff is cRaZy!
I'm sure this will help,

Return to posts index

Jesus Diaz
Re: Tracking a 360 footage - Oculus Rift
on Sep 1, 2015 at 7:51:02 pm

Hi Simone, Just wondering how you were able to pull off the track motion for the 360 footage you were asking about? Did you end up using sky box?

Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2018 All Rights Reserved