I wanted to document this issue in case someone else has the problem.
I authored a single-layer SD DVD in DVD Studio Pro. I wanted to create a DDP for replication, and I prefer to use DVDAfterEdit for that, as I also use it for other purposes. It's a nice program for changing things "under the hood" to which DVDSP doesn't give access.
DVDAfterEdit (AftEd) asks for a VIDEO_TS folder and works on that. When I selected the VIDEO_TS folder for my latest DVD, AftEd said it was a dual-layer disc. It's less than 3.8 GB, and is not a dual-layer project. Facing a deadline to ship to replicators, I needed a quick solution.
I tried deleting the VIDEO_TS folder so DVDSP would have to build from scratch, and not keep some (possibly damaged) files. AftEd still reported it was a dual-layer project. I removed the layout files from the folder so AftEd couldn't refer to them (guessing that DVDSP had placed incorrect information in them). That made no difference.
WORKAROUND: I had needed to burn a DVD anyway as a check disc for the client, so I put that in the drive & pointed AftEd to the VIDEO_TS folder on the physical DVD. AftEd reported it as a single-layer disc (it could hardly argue otherwise), and I was able to build a DDP from that (albeit more slowly than doing so from hard drive files). I prefer to build from the hard drive files, as the burned disc may not be a perfect digital copy.
I've since checked AftEd on other projects, and they display correctly as single-layer discs, so the issue appears to be with DVDSP on this one project.
I hope this workaround can help someone else sweating bullets.