I don't think that a man who is turned on by sex with transexuals(not transvestites) is gay. If he's attracted to both normal women and transwomen but not normal men, he might not be classified as gay or bi. Some might like them because the like their appearance as women.
I've never had sex with trannies, I've just watched tranny porn.

When I was young (like primary school age) I didn't even know what a womans genital looked like, I assumed it was like that of a man, until I had a sister. I had secret access to porn at the age of 12, I liked heterosexual and lesbian porn, when I took a peak at gay porn I was disgusted and I quickly left the site, but when I started watching shemale porn, I realised that my imagination(women with penis) was true. As a 17 year old, I discovered that transwomen weren't born the way they are, in an embarassing way. While people who know about it were calling me gay/bi, I found it as an insult and took it that I'm straight.