AE isn't a 'realtime' programme, so you will always have to wait for previews to render out regardless of the number of clips in your comp. The more clips you have, the longer it will take to generate a preview, but I don't believe there are any hard and fast rules about how many are too many. One way of working faster is too toggle off the visibilty for the layers you're not working on. For example, position the first video clip in your video wall, then shut if off and bring in the second, and so on.
You might also consider using proxies if you have high res files, or setting the comp resolution down to half or lower.
A further point: if you're using compressed video, e.g. mpeg, AE will take longer to generate a preview because it has to 'create' the missing frames as needed. Uncompressed video will probably be faster.
Simon Bonner