I agree that they don't have to be gender-specific, but in my experience, in the great majority of cases, they have been used as such, and I think most people still perceive them as such. That is why they bother me. I also agree that they aren't always used as slurs - certainly at least within the BDSM community, they are often used in a positive sense. But still, in this case (in regards to the negative comments the original poster received), they were definitely being used as slurs.
no subject