To say golf has a literature is not to say golf writing is literature. The quality of any such literature is more to do with the writers and not the sport; however, in the U.S. only baseball in quantity and boxing in quality can compete with golf as a sport for its attraction to top-flight writers.
Grantland Rice, Rick Reilly, Herbert Warren Wind, Tom Boswell, Leigh Montville, Frank Deford (a non-golfer whose 1998 diatribe against golf is a personal favorite -- we take what we can get from the greatest sportwriter who ever lived, or lives), these are but a few. You will know of more.
Furthermore, it's important to remember that so much of the writing is done on deadline, and this includes even Wind, although the old New Yorker "deadlines" give the lie to the term.
As far as writings I think non-golfers would enjoy -- for I enjoyed many of these before I became a golfer -- a short list:
Rick Reilly's game stories on the 1986 Masters and on the 1995 Open Championship, written on deadline, have held up very well, as have Tom Boswell's pieces on the mental and physical architecture of ANGC -- held up as an evocation of a lost architecture, that is.
As far as essays, those published by John Updike (who wrote the best sports essay in U.S. history), David Owen (especially the first essay in "My Usual Game"), and George Peper. All share the sublime and too-rare character of self-effacing humor.
George Plimpton did as well on golf, which is to say very good, as he did on other sports. Impossible not to hear "Japanese admirals" and not smile. I confess to hot-soaking my hands before a big match, too! In the category of participative journalism, John Paul Newport's entry is another funny read.
Dan Jenkins deserves to be called the father of deadline golf writing -- and his novels will confuse no one with literature, least of all him, yet they are equal in quality to, say, the works of top-tier crime fiction writers. (Let's remember that like the best sportswriters, he was drawn to golf but not only to golf -- "Semi-Tough" to offer the best example.)
And speaking of novelists, let's not forget Wodehouse, who still finds an audience of non-golfers.
There's a ton of dross out there, but that's true of everything, not just writing.
Mark