A Famous Study Found That Blind Auditions Reduced Sexism in the Orchestra. Or Did It?

[Chris]

The tendency seems to be for science to slide into politics--perhaps slide back into politics. But on occasion falsification rears its ugly head.

A Famous Study Found That Blind Auditions Reduced Sexism in the Orchestra. Or Did It?

One of the best-known scientific studies to posit that implicit bias—the idea that all people are unconsciously racist, sexist, etc.—can be counteracted via strategic effort is taking a well-deserved beating. It now appears that the findings were significantly overstated.

The study, "Orchestrating Impartiality: The Impact of 'Blind' Auditions on Female Musicians," by Harvard University's Claudia Goldin and Princeton University's Cecilia Rouse, was released in 2000. Its bombshell finding was that blind orchestra auditions—which prevented the choosers from seeing whether each auditioner was male or female—increased female auditioners' odds by 50 percent....

...in an area of research fraught with replicability problems, the orchestra study was supposed to be one of the good ones.

Well, so much for that. In May, Columbia University statistician Andrew Gelman took a deep dive into the study. He described them as "not very impressive at all," and had great difficulty trying to locate the 50 percent statistic within the modest findings.

"You shouldn't be running around making a big deal about point estimates when the standard errors are so large," he wrote. "I don't hold it against the authors—this was 2000, after all, the stone age in our understanding of statistical errors. But from a modern perspective we can see the problem."

...Blind interviews and auditions may be preferable for other reasons. They may even reduce implicit bias in some situations. But as is so often the case, the sweeping claims of social scientists do not seem to survive scrutiny.

It will be interesting to see if diversity coordinators—many of whom incorporated this shoddy scholarship as part of their training seminars—adjust course, though I wouldn't count on it. The entire concept of microaggressions lacks scientific legitimacy, after all, but this hasn't stopped college diversity czars from policing them.

[Marcus Aurelius]

As a classical musician with some experience in symphony orchestras, I can speak to this subject. For starters, it's true that professional orchestras were male-dominated throughout the 19th and 20th centuries, up to about the 1960's. But I believe that that was a reflection of societal gender bias, not something unique to the industry. Women were traditionally encouraged to take up instrumental performance only as an avocation, a finishing school enhancement if you will. Few women pursued enough education to compete with the top-tier performers. But it's been known since the dawn of Western music, that women are every bit as capable of virtuosity as men, and there have been some famous women instrumentalists: Maria Anna Mozart, Clara Schumann, etc. (See a list of renowned 19th century women musicians here. It's longer than you might think.)

Cue Women's Lib (which actually began with the suffrage movement) and over the course of the 20th century, women start pursuing the highest levels of education in performance and begin making their way into A-Level orchestras. Now I don't know how the behind-screen auditions originated. It is possible that some people thought it would create gender-blind adjudicators, but my understanding has always been that it was an effort to avoid nepotism. The community of classical musicians that can play at an A-orchestra level is such a tiny fraction of the populace that everybody knows who everybody else is, even if they haven't met them. Trade journals and recordings keep everyone apprised of what everyone else in the business is doing, and who their students are and what they're doing as well. My understanding has always been that the screen auditions attempt to negate cronyism.

More to the point, however, is that the screen test is only a preliminary. In the final audition, the applicant plays in full view of the search committee (comprised of orchestra members) and the conductor. So if the adjudicators really wanted to engage in any sexism (or racism or nepotism for that matter), they could easily do so then. (Which makes the screen totally pointless - honestly, I don't know why they bother with it anymore, except perhaps because it looks politically correct?) I agree with the commentator on the video, that looking to attribute the current gender parity to the screen test is far-fetched. Women increasingly became a part of the symphony orchestra industry simply because, over the second half of the 20th century, many more of them were educated to the necessary level of virtuosity.

[Helena]

That was interesting. Thank you. I hadn't given any thought before to the small community aspect that may have contributed to the perceived need for a screen test.