Archive for the 'Statistics' Category

Boys, Girls and Hot Hands

This is a post about hot hands in basketball. But first, some relevant history:

The single most controversial topic ever broached here on The Big Questions was not Obamacare, or tax policy, or the advantages of genocide, or the policy treatment of psychic harms. It was this:

The answer, of course, is that you can’t know for sure, because (for example) by some extraordinary coincidence, the last 100,000 families in a row might have gotten boys on the first try. But in expectation, what fraction of the population is female? In other words, if there were many such countries, what fraction would you expect to observe on average?

The “official” answer — the answer, for example, that Google was apparently looking for when they posed this as an interview question — is that no stopping rule can change the fact that each birth has a 50% chance of being either male or female. Therefore the expected fraction of girls in the population is 50%.

That turns out to be wrong. It’s true that no stopping rule can change the fact that each birth has a 50% chance of being either male or female. From this it does follow that the expected number of girls is equal to the expected number of boys. But it does not follow that the expected fraction of girls in the population is 50%. Instead, that expected fraction depends on the country size, but is always less than 50%.

If you don’t see why, I encourage you to browse the archive of relevant blog posts. If you still don’t get it, I encourage you to keep browsing. Whatever your objections might be, you’ll find them addressed somewhere in the archive. I’m not interested in relitigating this. I will, however, happily renew my offer to take \$5000 bets on the matter, on the terms described here. Last time around, all takers changed their minds before putting any money on the table.

Now let’s get to the hot hands.

Share/Save

Chain Reaction

If you study economics, or statistics, or chemistry, or mathematical biology, or thermodynamics, you’re sure to encounter the notion of a Markov chain — a random process whose future depends probabilistically on the present, but not on the past. If you travel through New York City, randomly turning left or right at each corner, then you’re following a Markov process, because the probability that you’ll end up at Carnegie Hall depends on where you are now, not on how you got there.

But even if you work with Markov processes every day, you’re probably unaware of their origins in a dispute about free will, Christianity, and the Law of Large Numbers.

Win Landsburg’s Money!!!

Last week I posted a little brain teaser that shows up frequently in recreational puzzle books — and reportedly in Google job interviews. The interesting thing about that puzzle is that the “official” answer is wrong. Not only that, but it’s wrong for an interesting reason.

I explained the official answer, I explained exactly where it goes wrong, and I explained how to get the right answer, citing Douglas Zare’s post here as inspiration.

The physicist Lubos Motl, however, still defends the official “50%” answer on his own blog. I am therefore offering to bet him \$15,000 that I’m right (with detailed terms described below). If you agree with Lubos, this is your chance to get in on the action. I will take additional bets up to \$5000 per person from all comers until such time as I decide to cut this off. You can place your bet by commenting on this post with the amount you’d care to stake. Be sure to include your email address (which does not show up in the post) so I can email you and verify that you’re for real.

Topsy Turvy

The AFL-CIO is calling for passage of the Paycheck Fairness Act to close the wage gap between men and women, a problem they say is increasingly urgent, with the above graph as Exhibit A. Get a load of that plummetting dotted gray line!

Now have a look at the right hand axis, which the perpetrators have conveniently drawn upside down for no apparent reason other than the obvious dishonest one.

The Match Game

Robin Hanson reports that success in marriage is quite uncorrelated with the match between your personality traits and your partner’s. Your traits matter (it pays to be happy, for example) and so do your partner’s, but the combination makes no difference. In other words, being a happy person (or an extrovert, or a stickler for detail) affects the quality of your marriage in exactly the same way whether you marry Ruth Bader Ginsberg or Lady Gaga. (This applies specifically to personality traits, not to religion, politics, wealth, intelligence, etc.)

Edited to add: The original version of this post misstated the result; I’ve changed a few words in the preceding paragraph so it’s accurate now.

From this, Robin concludes:

If you want a happy relationship, be a happy person and pick a happy partner; no need to worry about how well you match personality-wise.

NO!!!! That’s not the right conclusion at all, and it’s worth understanding why not. Suppose we lived in a world where personality matches had a huge effect on the success of marriages. In that world, why would two people with clashing personalities ever choose to marry? Presumably because there’s some special value in the match — like, say, an extraordinary mutual attraction — that overrides the personality clash.

So a survey of married couples — which is exactly the sort of evidence Robin is reporting on — is not at all a random sample of couples. Instead, it consists, for the most part, of couples with matched personalities on the one hand, and couples with mismatched personalities who are exceptionally well suited to each other for some other reason on the other hand. It’s not too surprising to find similar success rates in those two classes of couples. The third class — the couples with mismatched personalities and no redeeming match characteristics — never gets married and therefore never gets surveyed.

Conclusion: The results Robin quotes are perfectly consistent with a world where personality matching doesn’t matter — but also perfectly consistent with a world where it matters very much.