I am surprised or annoyed by this.

For goodness' sake, Harry, what are you doing at this time of night?

Well, for goodness sakes, why didn't you tell me that earlier so I could have changed my plans?

Usage notes: used for emphasis and often used instead of the more offensive idioms for God's sake and for Christ's sake

Related vocabulary: for crying out loud

• take something with a pinch of salt
• takes the cake
• take ... stick
• take stick
• take stock
• take stock of
• take stock of something
• take the bad with the good
• take the bait
• take the ball and run
• take the ball and run with it
• take the bit between teeth
• take the bit between your teeth
• take the brunt of
• take the brunt of something
• take the bull by the horns
• take the cake
• take the cloth
• take the edge off
• take the edge off something
• take the fall
• take the fall for
• take the fall for someone
• take the fall for something
• take the fifth