To finish or cease doing something in a spectacular, great, or exciting manner. He really went out with a bang, scoring a touchdown at the very last second of the game! The whole point of celebrating New Year's Eve is so the previous year can go out with a bang!