Possible to track sky with single foreground object?

I wanted to replace my sky but I made the mistake of shooting from a low angle so I didn't get any buildings in the shot...only my subject and the sky itself (very low contrast sky).

I'm asking if anyone knows any techniques to solve for the background or camera using just the foreground, and a background that has no real track points (just very blurry wisps of cloud that mocha can kind of track but its a glidecam shot so they keep going out of frame).