I feel like being annoyed by something like hoodies is like being enraged by a bad print on a shirt, or joggers, or high-top golf shoes, or at one time visors, or thick ties vs. skinny ties, etc. Some people probably objected to quarter-zips because they weren't sweaters. Or when golfers wore full tweed outfits, someone objected to the first non-jacketed golfer, we can be sure.
Style changes. I may not like how the styles look, or want to wear them myself, but I try not to care what other people are doing.
Will I wear jeans on a golf course, or joggers? No, probably not. I have worn a hoodie because… they're comfortable. Especially Rhoback hoodies.