I am working on a scene where there is smoke moving in a bottle. I am having a difficulty in making the smoke appear like it is in contained within the bottle. Any recommendations how to make this look realistic?
The first thing you might want to do is, for your own knowledge, put some real smoke in a real bottle and study how it behaves. My first thought would be that the smoke, while billowing within the bottle, would exhibit hard edges where it hits the inner glass. So a hard mask - or maybe with the slightest feather would be necessary for the confines of the bottle. You've also got the thickness of the glass to bear in mind - that would be reflecting the smoke somewhat, but is probably small enough within the shot to not come into play. I think it's the motion of the smoke which is going to sell the effect, so by trying it yourself (maybe cigarette smoke blown into a bottle?) you'll see what you'll need to do. I would think that a couple of layers of Fractal Noise might do the job. Ideally, it would be done with 3D smoke, of course.
We did a "smoke in a bottle" effect a few years back in AE. There our solution was to shoot smoke at high-speed RED against black and add / screen it into the bottles. We had to work quite a lot with coloring the smoke, getting the correct level of "clean smoke" and not a feeling of dirty or hazardous smoke. We also worked with masks to confine it to a given area as well as using various displacement / disorts. Sadly I can't find a link to the given spot but the end result was very good.
If you can't shoot new footage I'd start to experiment with a particle system. From my experience these always end up more fake than the real deal though unless someone really experienced is working the magic. Subtitle smoke tends to work this way (I've done that my self) but I always recommend shoot real or getting stock footage.
I agree with Erik - a practical solution will always look more real, although good masking would be essential. I think that either Fractal Noise or Particular could do the trick. Particular's smoke presets are ok, but definitely would need major tweaking to get the look you need. One other thought along the lines of my first post. If you have access to the bottle you used for the shoot - or a similar bottle, you could put the smoke in that bottle, and then try to camera-match the smoke to the bottle in the shot with masking and camera tracking. I don't mean to track the whole bottle into the shot - just the real smoke inside the bottle.
Thanks for all the suggestions. I don't have the original bottle but I think I have something similar and as luck would have it, this is the 4th of July weekend so I am pretty sure I can find someone with a smoke bomb. If you folks come up with anymore ideas or find the link to the spot that was produced please let me know -- thanks again!