there are a several ways to achieve what you want but it's not going to be that easy. you need to have this shot without your guy and cleaning it requires tracking the footage, cleaning the guy from it, and attaching it back as a clean background. since your camera is moving and you have perspective in your shot, it's going to complicate things even more. now when you have a guy on the background on one layer, and the background clean on another - you can decide how exactly do you make him disappear which is relatively easy.
there attempts to pull this off come to mind:
1. you can try Mocha Remove module to remove the guy:
2. another more straightforward approach would be taking a few frame from the footage with the guy in it, cleaning it very carefully in photoshop, and returning it to Ae and attaching the clean background to the footage via 2D tracking. you would have to track different areas because you have perspective in your shot. the tree and the floor would be two of them.
3. another approach would be same as 2 for cleaning the shot, but to use camera projection to project the clean images on top of a 3D tracking data. like in this example:
