if you're just doing image slices, then css won't be of much use to you maybe outside of positioning. css is used to change styles (bold, italic, etc.) and position elements (absolute or relative to each other)...the div tag is just a tag that contains content that you might want to apply styles to - same with span...
but if you're just doing tabled photoshop elements, then i wouldn't think you'd have a lot of use for it for those types of designs. a perfect example of where css comes in handy would either be a text header or body copy for your site - you can assign styles to those and then you can just wrap the content in a div or span or table or whatever and give that element your style class and then it is good to go with your desired layout style.
The menu on the right is for the most part all CSS. The must use a graphic of some sort for the little page triangle, but I'm sure it's just loaded once. The menu also stays in place when you scroll. Jump around the CSS tips and tricks on that site and there's some neat stuff.
for tutorials on css, it's best to just hit up google - i could write up a full explanation of the div tag here, but i'd just be doing what a ton of other sites already have... :-/
a div tag by itself is relatively useless. think of a div tag as a place to make use of css. things you can do with css are (on a high level):
- placement of items (including margin and padding)
- style of items (font type, font weight, etc.)
- colors (background, foreground, images)
that's very high level but it gives you an idea. what bret was saying (at least what i got he was saying) is that with css, you might be able to replace some of your slices with css-controlled text content, which will load much faster. in that case, you would need to pick up some css knowledge.
so for both that and a general understanding of css, you can either try that link bret gave or this one is pretty good too: