Why Does Everybody Always Look Surprised at the Elevator?

Unless I'm popping down the hallway to my trash room, LITERALLY anytime I leave my apartment I have to take an elevator. A conservative guesstimate would put me in an elevator around 112 times in a single week, every week, and yet I haven't watched a single person about to step into the elevator who didn't look 1000% surprised there was already someone else there.