Fair. There is no accounting for taste. But you're missing out on hilarious characters like Agent Smecker from Boondock Saints—a HOMOPHOBIC gay man:
Agent Smecker: "What are you doing?" (after bitchslapping dude)
Rando gaydude: "I just wanted to cuddle."
Agent Smecker: "Cuddle? What a fag."
(if you can stomach the scene lol: Boondock Saints | Willem Dafoe "Cuddle?". If only he had said "Lol, fag" instead). EDIT: That's a good movie in general if you like movies that are unapologetically politically incorrect and proudly about being a man.
The other thing you might you might be missing out on is some troubled protagonists when excluding that kind of thing (feeling of isolation or self-loathing are, IMHO, really fun foils for characters, and gay/etc characters are a group that can fit well that sort of thing.)
BUT... in the world of ameteur fiction, a lot of stories centered on sexuality or sex in general are written by teenagers or gooners trying to living out their fantasies, rather than interesting stories about people who happen to be gay. So in that sense you're probably sparing yourself some trash.
The good news for you is the overwhelming majority of stories here are with either straight characters or sexuality is irrelevant.