This could be a problem with your DVD player (it's only outputting 2 channels) or with the Nuendo encoder.
Are you playing the disc on a set-top player or on a computer? Try both, and try multiple players (set-top or software). If it's a software player, it could be a limitation of the player (ie, the publisher of the player software wants you to pay more) or your sound card (does it have 5.1 channel capability?). If it's a set-top player, make sure you've got it connected properly to a 5.1 channel decoder.
As for the Nuendo possibility, I can only suggest using another program to check the file, but I don't have any programs to suggest. Maybe look at VideoHelp.com for an AC3 file utility?