I'm working on a project using Maya 8.5, and we are using a Win XP Pro Core 2 Duo Extreme system. It has 4GB RAM, 1 TB HD space, and an 8800GTX video card.
Our master scene file (the one from which all the animated shots will be made) contains about 2.2 million faces, about 45 shaders, and is approximately 115MB in size.
I can open it in about 10 seconds in wireframe mode, but it crashes the moment I switch to any shaded mode.
Is there a way I can troubleshoot this so that I can get work done? Any ideas are welcome except for upgrading the video card (I work for Intel, so replacing the computer itself is easier than getting money for a Quadro).
one thing you might want to try is to set the OGL texture size to something lower. I'm not in front of maya right now, so bear with me.
Some where in the display preferences there is a setting to tell what the default texture resolution it loads to the video card is. Set this to something reallly low. I remember a scene a year or two ago that had this problem (big scene, lots of characers, crashed as soon as it went into textured mode). It sounds like your overwhelming the video card memory resources. If it does it in non-textured shaded mode too, i'm not sure this will fix it, but it's worth a try.
let me know if you can't find the setting i'm talking about.