People buzzing by me never piss me off... they just remind me of how many people are so horribly bad at math (e.g. 10mph over the limit on your daily commute, if you could even sustain that [and with red lights and stop signs, that's nearly impossible], would only save you about 5 minutes, on average, per DAY). Obviously, if you're doing a much longer trip, the time savings could be significant.
What pissed me off is that they probably freaked out that I was slowing down which makes for a dangerous situation. Then, on the other side of the town, they didn't feel like driving the 75 mph speed limit and I would have to pass them.