Partagé en mode public  - 
 
Priors and double standards

So a line in Ta-Nehesi Coates' piece about Obama last week--saying he (and other black people) had to be "twice as good" to get to the same place--got me thinking.  It seems on first examination that double standards are totally unreasonable.  Switching gears to a similar area where I feel a little bit more confident in the data, I've long thought there's a lot of merit to the hypothesis that inequal representation of men and women in advanced science is due to  greater variance in male ability (hence more outliers).  But one of the few pieces of hard evidence (as opposed to a lot of namecalling) its opponents have is there are, in some cases, double standards for men and women.  The canonical study gave subjects the same resume with male and female names and got back higher evaluations of the men.  This seems, if true, pretty bad on first grasp.  There's some fixed talent level which is Good Enough for any job, we'd think, and it doesn't matter who you are (or how rare someone like you is) if you're above that bar.

But over the weekend, I did some more careful thinking and a bit of math, and came up with something interesting: in some cases, double standards are precisely the rational response to uncertainty in evaluation.

Let's say I'm hiring engineers, whose skill can be measured by a real number S.  S is distributed N(0,s) in a population.  To be good enough for my company, I want engineers with S >= 3, and as usual in these situations, type I errors (hiring someone not actually good enough) are worse than type II errors (not hiring an adequate candidate.)

But I can't just look at someone and read off their S-value.  I have to interview them, a process which has some error. We'll say interviewing is unbiased in the sense that my observation of skill from an interview (O) is distributed N(S,t) (a bell curve around the true value.  It's equivalent to say O = S + E, where E is a N(0,t) independent error term.)  Now, how should O make my decision?

The natural answer is by looking at the distribution P(S|O).  It's not hard to show (conjugate prior) that this is distributed N(O * s/(s+t),st/(s+t)); a bell curve, centered at a weighted average of O (my "unbiased" guess for skill) and 0 (my prior best-guess.)  Now suppose I want to have at most a 0.1% chance of accidentally hiring someone with S < 3; where do I set my bar?  I'll save you some Mathematica time and tell you the answer is (approximately, and I think--anyone double-checking my algebra would be appreciated)

B = 3(s+t)/s + 3.09023 * sqrt((s+t)t/s)

Let's plug in numbers.  We'll take s = 1, t = 0.1.  Then our bar is 4.32.  This makes sense.  When you tell me "Candidate is >= 3" you're asking me to believe a 3-sigma event.  If it isn't at least a 3-sigma event that your test error could put his true skill below 3, I'm going to lean towards that explanation; since I want an overwhelming chance that true skill is >= 3, I want an even larger margin, here 4.17 sigmas of testing error.

But suppose those are the numbers for men, and women have lower population variance: s' = 0.55 (thus a standard deviation ~0.75 of the male value.)  Our new bar is 4.76; you're asking me to believe a 4-sigma event; I need 4 sigmas of testing error to even consider the hypothesis and about 5.5 to be pretty sure it's the right explanation.  A double standard, to be sure, but in the model here, it's exactly the fair thing to do.  If you ask me to believe something less likely (in prior) I need stronger evidence.

This equally well deals with situations like differently evaluated resumes: if we taking reading resumes to provided an unbiased estimate of S, the prior still shifts back my max-likilihood estimate of true skill by the same ratio of variances.  So my best-guess for true skill of a woman (in this model) is less than that of a man given the same observables.  We can't ignore the prior unless our evidence is totally errorless (and note that as t goes to 0, the above expressions converge to values independent of s, as they should.)

This is not to say that all double standards are necessarily correct.  If we take Coates literally, and we ask black people to be "twice as good" (according to our observation), we either need the variance of testing to be approximately the same as population evidence (absurd), absolutely huge variance differences between black and white people (no evidence whatsoever), or a several-sigma difference in mean (also unlikely.)  But we do need to recognize that not everything that seems unreasonable at first glance--treating the same evidence differently in different cases--is actually unreasonable.  We cannot ignore our priors.

And I'm significantly lowering the weight of the resume studies in my evaluation of gender gaps in science.
Traduire
1
Photo du profil de Yllona RichardsonPhoto du profil de Phil MillerPhoto du profil de Andrew HunterPhoto du profil de Aaron Wood
32 commentaires
 
Except that, even in cases where performance can be judged anonymously on the spot (auditions for symphonies, for example) - under varying times and conditions - after the curtain comes up and the musician (who was selected by male musicians on the basis of what they heard) is revealed to be a woman, the boys still try to kick her out of the club house.

Also, you're attempting to apply mathematics to something that isn't readily quantified. You would have to eliminate the subjective element (the human interviewer) entirely. Probably, you would need to use a multiple choice test, graded by a completely impartial observer. (I suppose you're trying to adress that by pointing out that we can't ignore our priors, but that rather is the entire point of the complaint in the first place.) The interviewing process is notoriously useless for anything but the human factors - "Are we going to be able to get along with this person?"

The complaint around institutional bias has to do with institutional fairy tales that authority figures told for generations. The fairy tale that African Americans were more prone to criminality, for example. Or the fairy tale that women aren't as good as math (that was the beginning for the end for Larry Summers - http://en.wikipedia.org/wiki/Lawrence_Summers#Differences_between_the_sexes - at Harvard).

The bottom line is that all people want to be viewed as individuals and not as part of some "mass," particularly when it comes to something as idiosyncratic as their skill sets and the jobs they aspire to do.
Traduire
 
Except Larry Summer was right.  Go back and look at how he got kicked out--a few professors complained that his comments made them sad and angry, not that they had any evidence he was actually wrong.

I also disagree that you can't quantify intellectual skill.  Pretty much every major company, school, and other group that recruits smart people does too.

I agree that we should treat people as individuals, but when you're claiming to be an atypical individual--an outlier--you need to present evidence to that effect.  If I tell you I can read minds (and do some cold reading on someone) do you take me at face value as an individual, or note that the vast majority of people who claim to be psychic aren't?
Traduire
 
I take it that you've read the book by the name Outliers? You might find it interesting. He deals with the whole morass of quantifying success and makes some interesting points about certain myths we have about hard work, success and intelligence. I also find it interesting that that you've moved away from sex (female v male) and toward specific claims of skills in your last paragraph. If there were a recognized, objective means of verifying said skill, then yes, I would expect you to score well on said evaluation if you so claimed.

Summers was wrong, actually. He failed to look at other factors - and the few he did cite are actually what is wrong with industry and capitalism as it is currently practiced. He was condemned for careless scholarship, in fact. 

Among the factors Summers missed: You can walk through any CS class at a major research university and your nose will give you one explanation as to why there aren't as many women in at least that field. When you combine that with other factors (such as social awkwardness, insensitivity to others, etc, etc - none of which are malicious or conspiratorial, by the way, but which are amplified in the presence of members of the opposite sex) the classroom environment in the hard sciences becomes unpleasant to women. Generationally, there are fewer barriers to entry now than there were in, say, the sixties (when women were told by their advisors to seek more "appropriate" fields when they expressed an interest in the sciences or medicine). There is still that something about the hard sciences that attracts people who are, well, to phrase it diplomatically, different.

As the number of women graduating from universities outpaces men, as engineering becomes less a specialist and more of a mainstream skill (already, there is evidence of commodification) I would expect more "normal" folks to get into those fields. Once that happens, you will find - I speculate, of course - that the variance will begin to collapse to the point where it is negligible.

And I stand by my point that the interview as a hiring instrument is useless for anything but personality evaluation. Tests - as in clinically researched, statistically proven evaluative instruments specifically developed for the task - are probably helpful for evaluating claims of skills. But the interview is just to determine if the person you're interviewing is going to burn down the building or not.
Traduire
 
I think the problem with this is the fact that much of human decision making is not rational. In fact it is one of the classic problems many mathematicians make about behavior, is a rational actor assumption. There is also the tendency to inject ones own biases, though this is a problem when addressing any human subject.

 To take on any social science, whether it be economics, political science, or sociology, one must enter with the knowledge that most human beings are irrational actors, and only a few make decisions on a rational basis. Additionally there is a large risk of to place ones own biases into the theory, which creates problems, and often ignores critical externalities. Upon that you have to remove the factors which drive additional irrational decision making.  One has to make the assumption, that they themselves are probably irrational in their evaluation  When one is using such mathematics, it is often ignoring these critical factors for the sake of a more expedient formula, whether it be institutional discrimination or pattern matching biases. The problem with the formula itself is there is a level of pattern matching bias written into it.  This largely reinforces negative externalties in human evaluation. Often it reinforces such biases that are already in place. Again, properly taking into account such external factors, and addressing them in a way that is neutral, is alone difficult to do.  Privilege and pattern matching itself often perpetuates itself into assumptions. 
 
Much of bias comes from what is essentially a very human response away from rationality. Much of pattern matching bias is not from those in the select pool of actors, but often an emotional response for those more like you, and rationalizing such an emotional bias. Essentially, empathy is more likely for those who match your pattern, and it becomes integrated into the reasoning.  One calculates the existing pattern based on their own assumptions, rather than addressing the underlying issues involved in the pattern, and figuring in external factors. It is a surface level analysis as supposed to a deep level one. One has to take into account, ones biases are in no way rational, and may in fact seep into ones work. This is classic problem of pattern matching bias. The goal is to remove such biases, and to reach a more neutral point of evaluation that allows for rational evaluation, rather than working such factors into the evaluation. This pattern matching bias issues has long been a classic problem for behavioral and social sciences, and one that is often ignored when those outside the discipline try to engage in behavioral and social sciences. It is a trap that they often fall into when trying to explore theories involving either. As a result the patterns themselves and what one sees may be self-reinforcing with since biases sometimes remain in place, if not reinforced.  

What they are finding with the differences between men and women with advanced sciences is not greater outliers (that alone is an engendered assumption), but a mix of pattern matching bias, institutional sexism, and what they are finding out more to be true, is an underlying stereotype threat in the advanced sciences. This is an issue that is being studied extensively, and sometimes the issue can be addressed through a change in policy, and other times it requires a culture change, which is much harder to institute. 

Race, like gender, is another factor, one can look at lagging factors in education, but the number of externalities that are at play are significant, and often cultural and institutional. A form of stereotype threat probably exists there as well.  The point being is it is not so easily generalized on one factor, where there may be underlying issues at play as well.
 
Traduire
Traduire
 
re: "unpleasant environment" -- putting aside the childish hygiene slur, which I find personally and professionally offensive, the last few studies I saw on the subject could be summarized as "geeky people have a culture, which women tend to dislike.  It must therefore be bad and eliminated."  This seems a bit specious to me.    But you yourself say that hard sciences attract people who are different, abnormal.  I don't disagree.  In fact, the numbers suggest that on pretty much any axis, men are more likely to be farther from the typical value.  You're arguing my side here.

re: interviews are useless - you clearly have not had a technical interview, which I don't blame you for (my family of English majors gave me a lot of similar advice, not understanding what this different world is like.)  It turns out to be not unlike an interactive direct test of skills; it is entirely unlike a nontechnical job interview.  If you'd prefer to make arguments about objective tests, that's fine too--the same argument applies to IQ tests or similar (except the testing variance is likely to be lower.)
Traduire
 
+Christine Paluch Yeah, not everyone is a rational actor (though we can aspire towards that goal...)  This doesn't mean mathematics don't work. I also strongly dispute your claim that we can't make objective measurements of mathematical quantities because some people involved don't think fairly.

You should be aware that there's no evidence stereotype threat exists under high-stakes testing; in other words, it's possibly true when you ask people to take meaningless tests, but to our best knowledge, not when they actually care about the results.  
Traduire
 
Okay. Start out by presuming your facts. Iterate this process, feeding the results of the last evaluation into the next evaluation. Where does the four-sigma female employee end up? 

(Incidentally, given iterative trials, this is a process which might produce substantially better results as a whole if the approval process were fairly noisy.)
Traduire
Traduire
 
Innovators need perspective, and that's hard to get when everyone comes from the same background and has a similar experience of the world.  

This whole discussion seems pretty narrow, like there's an obvious and objective measure of talent in math or science or whatever.  It's not like the entire point of investigation and engineering is to max out knowledge and IQ.  We want to make new and better things that we haven't even considered before.  (Plus, I'm getting hints of scarcity here, like we don't have nearly enough smart people doing smart things.  That seems pretty unlikely to me.)  
Traduire
 
+Andrew Hunter So, okay.

Over the course of a career, a particular employee undergoes a number of evaluations for particular positions. Estimations of their capacities aren't simply based on a de novo evaluations of their capabilities; they're based, essentially, on the running average of their evaluations over prior positions. Which is why you don't get eminently qualified people from the mailroom, if such people exist, being hired to the boardroom. 

So, let's say we have a heuristic which evaluates a particular level of capability as three-sigma for women, but four-sigma for men. On a single trial, given your facts, this might work. (I also disagree with your facts, but I don't think those can be resolved here.) But over multiple trials, given a pass-fail evaluation (e.g. 'got the job' v. 'did not get the job'), the effects of that skeptical attitude are amplified with each trial. 

Given that there are a very small number of highly-selective, resume-dependent positions, and that credible applicants for those positions must undergo multiple trials before actually taking the position, it isn't obvious to me that highly-competent women would not be weeded out of competition for highly competitive jobs by a heuristic which otherwise makes sense (on your facts) for single trials.
Traduire
 
If the multiple trials are considered to be independent, this actually doesn't hold (in the model--obviously we can construct a different hiring model with different conclusions.)  If I do 4 interviews with independent results in the above model, we can replace them with their mean and effective variance t/4; note in the above model that as t goes to 0 the gap in hiring bar goes to zero.  A more general posterior model is property 11 under Miscellaneous here: http://en.wikipedia.org/wiki/Normal_distribution

My estimate of someone's skill given a previous no-hire and a test is slightly lower than with just their test but not by a lot (I'll work out the exact value at some point if you're really curious); but the basic fact is that there is very little information to be drawn (if you're rational) from a no-hire; the prior probability given the above values and bars that we no-hire an arbitrary (male, for definiteness) candidate is ~0.99998. It's a very unsurprising event and thus slides our posterior estimates by a very small event.
Traduire
 
+Andrew Hunter Are the multiple trials independent? (E.g. "was not hired for selective job; chosen for less selective job on subsequent trial; submits resume containing less selective job to highly selective job; repeat.")
Traduire
 
+Andreas Schou In your model, here, no, obviously they aren't (because you constructed it to be so. :P) But seeing someone being chosen for a less exclusive job is a very, very weak signal against their competence in any reasonable model (as I point out previously.) It's a very overused signal in practice--think of all the college graduates that spend three years working retail because there are no jobs, and now are clearly unhirable--but in a fair world, if you get another interview, that should blow away its effect.
Traduire
 
Yeah. I suppose we'll find out soon how reliable that model of the world is.
Traduire
 
+Andrew Hunter - I'm curious as to why you find my referencing my own and others' experiences in CS classes to be offensive. I don't think you resemble that but I don't think I've met anyone in my 17 years in the industry who hasn't had that experience. (By the way, I'm a programmer, myself. And a systems administrator when required.)

As for the culture being unpleasant for women and must die, well...It's not just unpleasant for some of the women who find themselves in it. Fortunately, economic forces are resolving the worst of excesses in that people who can't at least "pass" as moderately sociable are driven out of the workforce. Off shoring seemed to force people into at least paying some attention to those around them. (And I've no patience for the but I have Asperger's! defense for asshole behavior...If you have that problem then any position requiring any sort of customer service should be out of reach.) And I don't think it should die (or even change) unless the people in it decide to let that happen.

As far as Gladwell goes, his points regarding being in the right place and time are accurate. (And how little that is mentioned in the American mythology of success.) Have you looked at how much a general manager at a McDonald's franchise earns on median? (They are better compensated than most programmers I've known. And most don't program or have a college degree.)

I would describe a technical interview as more of a test than an interview. And given the variance of techniques (some of which are outlined in a book called How would you move Mount Fuji?) I actually question them as anything other than a different sort of "these are the sorts of problems we solve here, how do you perform and do you like solving them?" type of general interview question. An interview process is a good expression of an organization's personality.

My objection to your hypothesis is that you're using it to take aim at women in the workforce. (And that it's the same sort of reasoning used by some people I grew up around to justify some of their views of African Americans.) Provided you're taking part in interviews, your positions in these matters will not serve you well should someone file a discrimination complaint against you. (Just an observation based on personal and others' experience.)

I'm genuinely curious - since I have seen several pieces from you referencing "feminist" commentators (quotes are mine) - do you believe that those writers are taking aim at individual men or the institutional issues that they are writing about?
Traduire
 
+Andrew Hunter Most of the interviews I have done well at were technical. They more focused on environmental, analytic, and statistical subject matter though. I actually prefer them, and I think they are better at removing biases, as they force people to focus on their knowledge or skills. I actually like them quite a bit. That is not what I took issue with. Though your assumption in some ways illustrates my point.

The problem I had was the bias and assumptions into much of what you originally wrote. There was a level of sexism, and pattern matching bias in some of what you wrote. 
The question to me is what the essential inflection points are with regards to bias, and to me it is clear it is the resumes. The fact is you are making an assumption of the skill of the interviewer based on gender, and previous patterns and assumed skill levels based on gender with regards to selection of the resumes. Largely because you make an assumption of like applicants skill based on gender, your own personal bias came out. It is at the core of problems of pattern matching bias on gender and race. That is what I took issue with. There was a clear engendered bias in what you wrote, and your selection there, an assumption of higher skill based on gender.

The technical interview itself does not seem to be the inflection point, that is based largely on skill of an individual applicant. To me it is the actual selection of candidates to interview, that is where there is a higher potential for bias based on sex on the part of the person selecting candidates. Much more of a possibility to fall into the problems of pattern matching with the resumes in the case you put forth, because you demonstrated it yourself. It is best not to make assumptions on a candidates based on gender, and you open the door to personal biases, until you actually interview them.
Traduire
 
+Matt Harmon _Mount Fuji_ is a well known source for bad interview questions.  Please don't take it as how competent companies do interviews. Not even Microsoft does that sort of gotcha puzzle (generally called "manhole cover" problems) interviews anymore (or so I hear; I didn't get any of those when I interviewed a few years back but I dumped them pretty early.)  I don't know where you work or what you do, but you really don't seem to have experience with the modern standards of Silicon Valley.  http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html is a decent introduction, if a bit old.

Personally, I don't think I'm "taking aim" at anyone in my views about finding talent.  I think a lot of people take aim at people like me and conclude that we're evil sexist racist people without much evidence, and that Occam's razor isn't a concern once you've decided there's a kyriarchical conspiracy, and I have a problem with that.   When much simpler explanations fit the facts, the burden of proof is on those alleging malfeasance.
Traduire
 
+Christine Paluch You've made a lot of vague allegations about "bias" and "assumptions" and "pattern matching" but not any actual falsifiable hypotheses about inaccurate decision making processes.  I'd refine that if I were you.  

You do have one point that's accurate: the easiest place to make mistakes is selecting interview candidates, because that's where you have the least information.   But this doesn't just apply to women turned down by evil sexists, it applies to everyone.  Male or female, a resume or job application is a fairly unclear signal with a lot of variance to it. Paul Graham points out that YC makes the most mistakes when deciding who to interview: http://news.ycombinator.com/item?id=3730691 Alas, like any employer, they have a finite number of interview slots (practically) and have to make the best choices they can, despite the large error bars.  Once you've gotten past that bar, most competent hiring strategies put vastly higher weight on the interview results than other factors, because of the lower variance in that stage.

I would agree that when hiring, I should, in theory, interview everyone.  Alas, I have a finite number of hours in the day, and that can't happen.
Traduire
 
+Andrew Hunter - I apologize. I'm not making any charge here nor alleging malfeasance.

But the bit that raises a flag in my mind is:

I'm significantly lowering the weight of the resume studies in my evaluation of gender gaps in science.

 Also, the allegation that I think cultures should "die" if women find them unpleasant. (I cited "unpleasantness" as one factor among many as to why I happen to think there are fewer women in the hard sciences than men. But honestly, I can't say I care one way or another.) And that, for whatever reason, your stuff in response to the gender trolls seems to enter my stream at precise times for me to notice it. (And none of this is an objection, just the series of factors that have made me curious about your positions on these matters.)

Having been the target of a discrimination complaint (that was later discovered to be a way for the woman making the charge to cover up her alcoholism and outright incompetence) I can say that once the charge is made, everything you've ever said or written comes into play. So my only intent there was to advise you on a possibility you may have not considered.

And you're right, I've no experience in Silicon Valley beyond what I manage to read in the media. Ironically, however, what I've skimmed of the URL you provided seems similar to the last interview I had (but I've not read it in detail).
Traduire
 
+Matt Harmon "lowering my weight" was shorthand for a mental procedure that goes something like this (vastly oversimplified; all P are personal estimates under uncertainty in the Jaynesian sense)

P(sexism is responsible for gender gaps) = P(sexism|double standards are real and bad)P(ds are real and bad) + P(sexism|!double)P(!double)

The calculation in OP lowers my estimate of P(ds are real and bad) (the "weight" in question; I now think that resume study is less of an indicator of real, bad double standards than I did) and thus my posterior estimate that sexism is at fault (since P(sexism responsible|ds real and bad) >> P(sexism|ds are either false or OK)).  Make sense?

why I care - I made a offhand comment on an
+Andreas Schou post a while back (which sparked a bit of a nasty fight) that the main reason I spend so much time arguing about feminism is that it's a fallacy I actually have to deal with.  I don't think it's the dumbest ideology I've ever encountered, but since I live in SWPL-land, it's a fairly common one. I can happily ignore Truthers or Birthers because they don't hang out in my social group or at work.  I have to listen to feminists, so I care about correcting mistakes there.

As for my own safety from allegations, well..."speak the truth, even if your voice trembles."  I'll take my chances.  
Traduire
 
+Andrew Hunter There are fairly simple ways to avoid bias in resume evaluations, and they often involve stripping the names and other indications of gender/race off of the resume. It is a simple thing that can be done that is very effective The problem of course is that it dehumanizes the process, and this causes some people discomfort, even though it gives them a more objective perspective on candidates.  However, it is more effective to evaluate candidates based upon their qualifications, rather than personal characteristics (whatever they may be). Automation obviously plays a role as well. Like any inflection point, there is a solution out there that can potentially resolve the underlying issues.

The results may not be perfect, but they will be removed from many personal biases that come from these types of evaluations. There may still be a pattern bias, but it is removed from many personal biases one may have. At least with regards to candidate selection. 

The problem lied when you take gender into account of weighing gender in  the selection of candidates.  By giving it weight you were demonstrating your own personal biases. That is the basis for a sex discrimination case, no matter what type of job it is. This is the thing that I would think is best avoided when evaluating resumes.

If you are wondering, the issue is what you said. +Matt Harmon is correct in saying there is more than enough here if you happen to have a sex discrimination or harassment claim brought against you. Not saying it will happen, but your own personal views do come into play with such cases.
Traduire
 
+Andrew Hunter - I have a better understanding of your positions now.

I would say that for the most part - and I say this as a guy who sat through several semesters of feminist literature (though I fear you may now discount my technical background, though I've programmed on a broad range of hardware, including PLCs) in college - I no longer subscribe to the view that any of that conversation has anything to do with me (despite my being a man and despite my experience) and I refuse to personalize it. (When I was younger, I did. Suffice to say that doing so did not serve me well.)

I do think it would be better if women were better represented in the sciences and I believe they will be within a generation (my own daughter, at age seven, is displaying good mathematical aptitude and interest) so I regard much of that as handwringing on both sides of the issue.

I have to say, I'm surprised that it still has enough of a presence to provoke people. I thought all that died out circa fall 2001. On the other hand, I guess every region, sector and company is different, so I can't comment on how valid  or invalid charges of hostile workplaces may or may not be.
Traduire
 
+Christine Paluch Discrimination law is broken anyway--look up Griggs v Duke Power.  I'm not saying I use the OP heuristics to evaluate anyone, just that it's not as "biased" as some would make it out to be.  
Traduire
Traduire
 
+Christine Paluch Ricci is a nice improvement but last time I looked I don't think (not an expert here) it fully overturns Griggs.  Remember the Ricci holding is that fear of a Griggs lawsuit is not sufficient justification to make a racially biased decision.  Not that Griggs criteria (facial neutral, disparate impact) aren't themselves enough to sustain a complaint of discrimination.
Traduire
 
+Andrew Hunter Your premise and this entire thread is fascinating. I've read everything through twice.

As an African-American woman with a couple decades of hardcore IT experience gone by, I beg to differ with your assumption, method and result.

However, the initial question you've attempted to solve via "mathematics": that "Blacks must be twice as  good, to arrive at the same place" is an everyday reality for me, and has been so my entire life. As such, my immediate response may be too visceral.

I will take the time to give this discussion the proper consideration it merits, and get back to you later.

Right now I am stunned into silence.
Traduire
 
Just to take things a different direction, I'll note (without doing the algebra) that with your model, if you had data on how differently male and female resumes were evaluated, you could actually estimate the evaluator's values for the variance in aptitude of male and female candidates. Of course, every evaluator's working belief of those values will differ, so you'll get some sort of distribution there.

Social scientists are generally expected to provide raw data from experiments to allow further analysis. Perhaps there's a possible easy follow-up for some of this.
Traduire
 
+Phil Miller That's an interesting idea, though you'd probably need to ask evaluators to give more explicit numerical evaluations--either in form of probabilities of success at some task or, like, estimated IQ?  And of course while my model doesn't consider it the evaluators might have different priors for the mean.  Cool thought.
Traduire
 
I was just going on a binary output of "do you believe the person described by this resume is not  S < 3 with p < 0.001?" or something to that effect. Continuous values would make each point more valuable, but that seems to me like a sample size problem, not an actual qualitative problem with the data.
Traduire
 
+Phil Miller Sure, yes, that's just less data than a best-estimate of S.  In particular, I remind you from my earlier comment that the prior probability of a true answer to that is something like 0.00002; whp if you do a reasonable sample size study of random population entries you're just going to get uniform no's across the board.  If you select your own set of "good" candidates, you've ruined the prior.
Traduire
Traduire
Ajoutez un commentaire...