An article by blogger Jay Jay on the Howl Sanctuary site stated of the Wee Review’s Fringe output: “it’s clear to see there is some [gender] bias at play”. It continued: “The Wee Review shows a strong bias towards men in the allocation of 5*” and “is more comfortable with female performers inhabiting the uncontentious 3/4* middle”. On a Facebook post she went further. “If you as a woman were only seen by the Scotsman and the Wee Review, that could ruin your year and potentially your career.”
This is strong stuff.
It is also pseudo-scientific tosh.
You can’t just tot up numbers and claim “bias”. That’s not my opinion, that’s well-established academic practice. There are standard, very boring statistical procedures to follow before you make any such claims. She is a Social Science graduate of the University of Cambridge (albeit one who had to “teach [her]self first Excel and then basic maths”). She knows this.
Her findings are tissue thin. Her article does not constitute proper, robust analysis. To wit:
- She did not scope out her study in advance and draw objective conclusions. She decided what she was going to find and wandered until she found it.
- She used her own narrow definition of comedy (e.g. in our data, she counts standup Hannah Gadsby as a female comedian but not character comic Pamela de Menthe. Why? No idea.)
- She didn’t use a full data set (only up 24 August)
- She had very small sample sizes
- She disregarded statistics that didn’t fit her theory (e.g. our average rating for women is higher than for men on her figures; we only gave one stars to men)
- She selected the only statistic that did (in our case, number of five star reviews)
- She didn’t do any significance testing (basic academic rigour if you’re trying to show bias)
- She drew the conclusion she had decided she was going to draw even though her limited, incomplete data, insufficiently tested, had produced very mixed results
- She offered no examination of the reason for the bias (which she hadn’t actually proved). She assumed it lay with the reviewer, when you would also want to explore whether the reason lay with something the performer was doing (e.g. men performing comedy that provoked extreme reactions at both ends of the scale)
- She examined bias by publication, but when offered data on individual writers she thought it “not relevant”.
- She dismissed all attempts to clarify and question her procedure and all additional input of data. To quote, “thanks also to the minority of men who took the time to kindly explain how I was wrong, you have to bless a tryer don’t you”
But here’s the rub. If simple totting up numbers is the way to prove sexist bias, she is condemned of misogyny by her own hand.
The Wee Review conducted an extensive analysis of the content of the Howl Sanctuary website.
Ruling out mixed gender bills, there are eight comedy reviews on the website. We broke these down by gender with the following results:
Men: 8 (100%)
Women: 0 (0%)
Next we looked at the comedy interviews on the Howl Sanctuary website. Again ruling out mixed articles, we broke these down by gender:
Men: 3 (100%)
Women: 0 (0%)
We conclude that “it’s clear to see there is some gender bias in play” at Howl Sanctuary. “If you as a woman relied on Howl Sanctuary for coverage, that could ruin your year and potentially your career.” The Howl Sanctuary is systematically excluding women in another damnable example of institutionalised misogyny.
This is of course patent nonsense. But it is the trick she pulled on us, and got Chortle to report on, the Guardian lead theatre writers tweeting about and dozens of social media friends agreeing with.
You cannot simply claim bias wherever you like. If you toss a coin twelve times, and it comes up heads 10 time and tails 2, you can’t conclude the coin is biased. If there are 10 men and 2 women on a bus, it doesn’t make the bus company sexist. You need to rule out chance, and you need to consider all the possible sources of bias.
So that you can draw your own conclusions, here is the data I supplied to Jay Jay in a good faith follow up e-mail, at considerable investment of time.
Our complete Fringe stats (up to 28th) based on anything that is listed as just “Comedy” or “Comedy” and “Cabaret” on The List‘s data. It excludes anything listed as “Comedy” and “Theatre”.
5* – 14
4* – 36
3* – 32
2* – 11
1* – 2
Average – 3.52
5* – 3
4* – 25
3* – 20
2* – 5
1* – 0
Average – 3.49
There are two 5* and one 4* drag comedian I didn’t include. You can include them under men if you like, to “prove” our misogyny, although it would be odd to base an accusation of misogyny partly on our appreciation of subverted gender norms.
- Gender breakdown of our writers’ list: 41 male (46%), 50 female (54%) although we had more men reviewing at the Fringe due to geography, work commitments etc. I offered to supply more info on this once I’ve compiled it. Jay Jay declined the offer.
- Our priority list of “must review” comedy shows: 41% female (average per publication: 31%)
- Our final review list: 36% female (average per publication: 31%)
It is my contention that none of this can be used to show statistically significant “bias” in our attitude towards female comedians. But examine it yourself if you like.
For context you can also view our full spread of five star reviews here, which include shows by white women, black women, LGBTQ performers, groups from all over the world, shows specifically addressing sexism and racism and sexuality issues. You can also examine our pre-Fringe interviews, which include, ironically, an interview with Lynne Parker, founder of Funny Women.
Pick it apart, analyse it, do what you will, but I trust that any reasonable person would look at that and conclude that we champion brilliant performers whoever they are.
Unfortunately, with witch hunts like this, you are convicted as soon as someone shouts “misogyny”. Accept the findings and you are guilty of misogynist bias. Challenge the findings and you are not accepting culpability, and thus demonstrating your misogyny. To quote Jay Jay on Facebook: “I feel that criticising my methodology does little to support claims of gender equality”. Consider that statement for a while. “Criticising my methodology”, i.e. basic academic practice, the means by which knowledge advances, equals not supporting equality. “I have produced a report that shows the earth is flat. I feel that criticising my methodology does little to support claims of wanting to advance science.”
There are masses of interesting things to examine about gender. It would be interesting to see how absolute performer numbers compare with numbers reviewed, for instance. It would be interesting to explore the breakdown by comedy types. Are women more likely to be musical comedians, and men to be straight stand-ups, say? And if so, why? And if so, does that matter?
For now, the most interesting table (and most robust data) she provides is her first one. Stars are distributed almost perfectly in proportion across publications. Within that there’s some variation – something may be going on at The Scotsman, and The Skinny if anything is over-compensating the other way – but there is nothing at all to justify the clickbaity headline. That’s not me, a misogynist man baby, telling her that, it’s her own data. If she wants to disagree, take it up with the entire history of statistical analysis. The actual, intellectually honest, headline to her work would be “research reveals no statistically significant evidence of bias in Fringe reviews”.
A good maxim of reviewing is “review the performance, not the politics”. A show may tackle some important subject, say the migrant crisis, but it could still be either a good show about the migrant crisis, or a bad one. It doesn’t get five stars just because it’s about the migrant crisis.
In the same way, people cheering Howl Sanctuary should consider what it is they’re cheering. “I’m dealing with this in Australia too” said one person on Twitter. What are you dealing with? The lack of provable bias in reviewers?
At The Wee Review, we are all unpaid. We spend thousands of unpaid hours supporting artists of all backgrounds, often at the expense of family and friends, and other outside interests. As an amateur publication, I’m proud of the breadth and depth of what we do, and the people we’ve got writing for us. I don’t have extra time to spend responding to anonymous bloggers tweeting us out of the blue shouting “bias”. But I do, because I care about these issues. Without question, there is merit in assessing Fringe reviewers for gender bias. Nothing I’ve said here seeks to deny that, nor the difficulties of women in comedy. But please, people of the internet, have some common sense, and if you really are going to assess websites on the flimsiest of data, accept that on their own terms Howl Sanctuary is the most misogynistic of the lot.