Well, it is possible that this might work, but we time the audio signal from the video clock. So my concern would be that without the video to clock the audio you might get some anomalies, that might cause the record to be accelerated or retarded by some amount. It could be a little (i.e. not audible) or could be a lot (very audible).
Needless to day we don't make a card specifically to do this. My recommendation here is that you feed some sort of video signal (even if it is just a VCR composite) to the board for timing.