Credits for Special Rooms are spent when you transition from the "Waiting for Players" screen to when the cards have appeared. If you exit at the loading screen, you'll keep your Credits.

Keep in mind that other players may be found quickly, and we recommend making sure you're ready to play before starting a game.