Gotta say the scariest for me was when I was one vacation once in northeastern Michigan.
We stay at these cottages that you can rent which are amazing(in fact I'm here right now, and will be until the 20th). So, about 3 years ago, my brother's friend Sam suggested we sleep over at his place. On the way to his cottage he constantly reminded us about how old it is and that his grandfather's ghost constantly haunts the place.
So, when we got there, since he didn't have enough beds, we all slept in the living room, which had three walls that were almost completely screen/glass windows.
When we finally got to sleep on the couches, Sam and my brother slept on the other side of the room and I slept on a couch in a corner made up of two screen windows.
Then a storm started whirling like a motherfucker. The house was like a wind-tunnel,making a ton of noise and effectively startling me. Then, the thunderstorm kicked in. Now, this is an old house. I wasn't sure if it could take this.
So, basically, I lay down on a couch, eyes wide-open, terrified that the house might collapse over my head, windows could shatter into my face, and of my friend's grandfather standing in the corner.