A lot of the discussion on the previous posts by Mark, Eric, and Mohan seems to tacitly assume that when we have a conference in field X, we dispassionately look for the best authors on X, invite them, they either accept and decline, and so on until all keynote slots are filled. Under this assumption, it seems sensible to ask whether a consequentialist approach is warranted here, i.e., is it OK to sacrifice excellent male speakers who are not invited if one of the invitees is a woman? But this dispassionate procedure does not seem to reflect the reality of deciding conference line-ups. Having been directly or indirectly involved in the organization of over a dozen conferences and workshops so far, I have noticed that organizers are not always, or even mainly, guided by the relevance of speakers for a particular topic. Instead, they seem to rely on things like hearsay, reputation, informal networks.
On the face of it, this is not unreasonable. Organizing a conference is a lot of work, time, effort and money. In my case, I have a fixed budget I can spend on all research-related expenses. So money I spend in organizing a conference, paying for travel of keynotes etc., I cannot spend on travel for myself, books and other research-expenses. If I invite a keynote from the States, that means one conference less I can go to myself that year. So it seems reasonable want to make sure our keynotes are reliable. That they show up and not cancel at the last moment. That they give a talk that is not extremely technical, dry or incomprehensible for people outside of field X. That they don't go over their (often very generous) time slot. That they allow at least some time for discussion. How do we know which people fit those desiderata? In many cases, we decide to invite people we have seen speak in the past, people who are part of our informal network, or people whose reputation we know indirectly through others. I'm not saying this is the case for all speakers in a line-up. But it's very rare that *all* speakers in a lineup are unknown in this way (i.e., qua speakers) to the organizers. This seems to disadvantage non-minorities in various ways.
Since women for various reasons don't speak so much at conferences as keynotes, they are likely not to be considered as future keynotes. On feminist philosophers, there was a poll on why this was the case: the main reason apparently is lack of funding. For instance, an event in the past two years I was invited to as a keynote did not cover traveling. The organizer did everything he could to get travel funding for the speakers, but could only get accommodation funded. As I'm an FWO postdoc, I have a personal travel budget, but many women are at places that simply have no, or very limited travel funding. And even in my case, over the past year, I had to decline another invitation for a keynote because it was in the States, and my travel budget was all gone. Tenured males are more likely to have larger funds for travel, since they are at more prestigious schools with large travel budgets. Even if not, their salary is such that they might be able to foot their own travel bills.
So you get a classic Matthew effect. If you don't get in the "keynote circuit" because of lack of funding, you are less likely to be considered for future keynotes. If you're not a keynote, you'll not get into the edited volume (many conferences have such volumes). If you are not in the edited volume, because you weren't invited as keynote, you do are less likely to get invited to events where volumes are used as a guideline for speaker lineups (I know people who use this as a way to see if they haven't overlooked obvious candidates for a lineup of speakers. The problem is, edited volumes are not put together in a dispassionate, unbiased way).
We cannot do anonymous peer review for keynote lineups (it's not impossible, but would require a radical change, for instance, anonymously review all contributed papers and pick the best 3 or 4 as keynotes). So, barring that, we need to think more consciously about the decision strategies we have for inviting keynotes. I've noticed that this is often an informal, fairly quick affair. We sit together at a table. We've picked the topic, got some idea for funding, and the time. One of us says "How about speaker A? I've seen him at a conference last year, and he was simply brilliant". A gets listed. Another says "B's very good too. I met her at a summer school where she gave a tutorial on topic related to X". Then we start thinking. Who else to invite? Ah, there's C. He was in that edited volume..." In a matter of between 10 to 30 minutes, the lineup is decided, and people are invited.