Mainstream barely means that there is a big trend going on. If loads of people listen to a certain band, they are mainstream. If many people wear the same style, it means the trend is mainstream. According to that definition going around, wearing black and stating “Your stuff is just too mainstream for me.” is indeed mainstream (lol).
Anyway, I personally don't give a fuck if the stuff I listen to is the most popular on or if I am the only one who likes it. Because it just does not matter. When you like something you should just go ahead and keep liking them. Anything else would just be biggotry.