To skip to the next replay, or to get back to the previous one, you can use Ctrl+[ and Ctrl+]
Unless you nean that those don't work on some replays, in which case, I'd suggest posting one or two of the replays that you can't switch away from, or can't switch to, in this thread.