I think the difference you would like to see made more clear is between "fandom is a female-dominated space" and "fandom is all women, all the time".
Fandom isn't all female, fandom is not inherently female. But it is a female-dominated space and many of us treasure it as a rare safe space online for women to talk frankly about the things we care about. In a lot of forums and venues, having a female screen name means you're 4 times more likely to be harassed. Women who talk about sex often get trolled relentlessly. Fandom is an exception to that and that's why some of us see it as a primarily female space, and see threats to it in gendered terms.
Re: stupidity vs. sexism
Fandom isn't all female, fandom is not inherently female. But it is a female-dominated space and many of us treasure it as a rare safe space online for women to talk frankly about the things we care about. In a lot of forums and venues, having a female screen name means you're 4 times more likely to be harassed. Women who talk about sex often get trolled relentlessly. Fandom is an exception to that and that's why some of us see it as a primarily female space, and see threats to it in gendered terms.