Tag Archives: Phil Arena

Stereotyping, and Rare but Important Events

Phil Arena has an interesting but problematic piece up at Duck of Minerva, entitled “Bayes, Stereotyping, and Rare Events.” The substantive topic of the post is a recent survey of Muslims that I’m not too interested in. But Phil uses statistics to mask a deeply flawed and irrelevant conclusion:

Put simply, the probability that you’d be mistaken to assume that someone who belongs to group Y is likely to commit or have committed act X simply because most such acts are committed by members of group Y grows exponentially higher as X becomes rarer. The reason you should not assume that a person is a terrorist just because they’re Muslim, then, is not just that this is politically incorrect and likely to offend delicate liberal sensibilities. It’s that it’s almost certainly incorrect, full stop.

The first and last sentences in that paragraph have almost nothing to do with each other. Phil’s conclusion is irrelevant, and the “full stop” leaves the most important part of the conclusion unsaid.

And Phil’s not alone in such a mistake. Take for example an recent statement on the NPR program “Tell Me More” by Fernando Vila. Fernando is responding to a statement that a disproportionate fraction of violent crimes in New York City are committed by African Americans:

VILA: Well, I mean, the notion of paranoia is a good one and Mario’s statistics actually sort of feed into that – into this culture of paranoia. I mean, the vast majority of black people are not committing crimes.

VILA: You know, it’s like to say, I don’t know – the vast majority of hosts on NPR are white males. That doesn’t mean that every time I encounter a white male on the street I assume he’s a host of NPR. You know, it’s just a backwards way of looking at it

Phil and Fernando make exactly the same mistake: false assuming the cost of a “false positive” (accidentally marking someone as suspicious) is the same as the cost of a “false negative” (accidentally marking someone as not suspicious). But the truth is all errors are not equal.

The cost of a mistake is a function of the severity of the mistake.

Is the cost to society of 1 false positive (falsely placing an individual under suspicion of terrorism) the same as the cost to society of 1 false negative (falsely removing suspicion from an actual terrorist)? No, of course not, but Phil’s post is based that on fallacy. Otherwise his conclusion makes no sense.

There is a serious question as to where we should become indifferent to the trade-off — 10:1? 100:1? 1:1000000? — but it is certainly not 1:1.

Likewise, Fernando’s statement on NPR is irrelevant. While the consequence of guessing an individual’s employment status at NPR might be 1:1 (few would care either way), the costs of falsely assuming someone would attack you is far less than the cost of falsely assuming an individual will not attack you. Again, there is a question of trade-offs — 1000:1, 10000:1, 1000000:1? — but the cost of all errors are not identical.

Now, obviously Phil and Fernando had different motives here. Phil’s obviously trying to popularize some basic statistics, while Fernando is doubtless ignorant of basic statistics. But in both cases an unwary audience will be led astray into thinking all errors are equally important.

Science and Steam

Reactions to two of my recent posts — Mark Safranki‘s excerpt of my review of America 3.0. and Phil Arena‘s comment on my post on antiscience, plus some twitter conversations with Colin Wight — got me thinking.

What is the relationship of Science to the great economic systems we’ve had — hydrological, steam-powered, and now whatever-comes-next.

Well, in a hydrological system you’re either at the Malthusian limit or quite good at killing people off through war of disease.

Science is too risky (might not work, might have bad consequences if it does work) to spend much resources on in a pre-steam, pre-industrial society. So you get a few intellectual giants shouting to each other across time — like the nameless Chinese inventors or named European ones — with relatively little utility within a human lifetime.

But once you have steam-power, and the economic system it enables, society becomes incredibly wealthy. So you get science, institution science, whether in the form of corporate labs, or academic science, or the Department of Agriculture. The methods of advancement are so different, and the pace of change is so much quicker, this Science in a modern science is a different beast from pre-steam science — natural philosoph– which was basically bored men every once in a while discovering something.

What comes after the reign of steam, and the industrial society? What does Science look like after the next transformation?

It will be exciting to find out!

The Place of Rational Choice

After criticizing Patrick Thaddeus Jackson‘s antiscientific and dangerous attack on Rational Choice Theory, I then turned around and attacked Rational Choice Theory itself for not being a scientific theory (though it can be a useful tool).

The lesson, I guess, is that simply having the right enemies does not make you right yourself.

My critiqued of both Jackson and Rational Choice attracted the attention of Phil Arena, both regarding antiscience and, more interestingly, regarding Rational Choice. Phil was kind enough to provide with me two articles, “Does Preference Cycling Invalidate “Rational Choice Theory”?,” and “Rat Choice Apologetics II” in which he had previously attempted to defend Rational Choice Theory from similar attacks.

Phil’s posts emphasize that Rational Choice is not a scientific theory.

The first post, on preference cycling, is an extended “just-so” defense of Rational Choice theorizing against laboratory falsification. Phil writes:

My big point here is that those who seek to justify a wholesale rejection of “rational choice theory” by observing that some laboratory experiments have found that some individuals exhibit behavior that appears to reflect cyclical preferences are overplaying their hand.

But Phil’s bigger points seems to be that any laboratory finding does not falsify Rational Choice, because some collection of mathematical formulas can be modified post-hoc to account for the behavior observed. This speeks to the cleverness of the Rational Choice theorists — like Freudians or Jungians, any observation of evidence of their model.

Rational Choice is like Interviewing, because just as no experimental result can falsify Rational Choice, no experimental result can falsify the feelings of an interview subject. Few who are planning a complex intervention would do so without interviews of one sort or another, and it may be that Rational Choice is likewise useful. But just as the interview is a tool, not a scientific theory, Rational Choice is a tool, not a scientific theory.

In the follow-up Post, Phil goes farther to protect not just Rational Choice Theory, but any implementation of a rational choice theory, from falsification:

Amongst formal theorists, there is significant disagreement about how to evaluate models in general. On one end of the spectrum, you have the strict interpretation of EITM, as espoused here and seems to be Morton’s preferred view here, though she does discuss other views. This view holds that formal models are important for ensuring logical consistency of theoretical arguments, but the value of these arguments is ultimately judged empirically. On the other, you have Primo and Clarke, who argue that there are many different roles we could ask our models to serve, some of which do not require any kind of empirical assessment. My own views, as I’ve indicated before, are closer to those of Primo and Clarke.

This is not scientifically serious. But Rational Choice Theory is not a scientific theory, so of course it doesn’t have to be. The purpose of science is to improve, predict, or control behavior (at whatever unit of analysis we are working), but the purpose of tools such as interviews, case study, and rational choice is to inspire scientists to come up with scientific theories that can make control, predict, and improve behavior.

Phil’s a clear writer, so his point is written clearly. And he’s write that science has certain requirements — such as predictive validity — that are as hard to get away from as Rational Choice Theory’s unfalsifiable assumptions:

When we evaluate arguments empirically, we make a huge, non-falsifiable assumption that the future will be like the past. Otherwise, it would be meaningless to claim to be testing the claim that X causes Y by observing historical patterns of association between X and Y. On a certain level, we all understand this. That is why folks worry about omitted variable bias with observational studies and external validity with experiments. But I’m not sure how many people really appreciate the depth of the problem.

But of course the difference is that the scientific requirement for predictive validity enables it to fulfill its mission of predicting, improving, and controlling behavior (at whatever unit of analysis we are functioning). Rational Choice Theory rejects the scientific need to predict, improve, or control behavior, because it is a “formal model” which are “logical consistency” and thus do not need “empirical assessment.” That is, Rational Choice is a form of “qualitative” (or better, investigatory) analysis, where mathematical equation balancing takes the place of interviews or subjective impressions.

Rational Choice has a place in science, like any investigatory or qualitative method (introspection, interviews, case studies, etc): to generate hypotheses. Rational Choice should be a part of science to the extent its scientifically useful. But like interviews, case studies, and the such, we can’t generalize from rational choice theorizing, but of course we can generalize from the empirical findings such theorizing might lead us to.

Against Rational Choice

I recently wrote two posts, “Four Types of Anti-Science” and “Academia, Science, and Anti-Science,” which took Patrick Thaddeus Jackson to task for his post, “The Society of Individuals.” I even criticized Phil Arena for not being sufficiently critical of Jackon’s writing in his post, “Should We Keep Hidden the Way People Behave When their Actions are Hidden?

But here’s the thing: I’m not a fan of “Rational Choice.” It’s a useful tool, but Rational Choice Theory is not a scientific theory.

For emphasis: Rational Choice Theory itself is not a scientific theory — it’s a tautology that’s used for creating theories, but it’s based on a basically absurd premise that is as protected from refutation as the worst nonsense from Sigmund Freud and Carl Jung.

The core foundation of Rational Choice Theory is that individuals have a discoverable complete transitive preference schedule. This is a ridiculous assumption. It’s also unfalsifiable in terms of the theory that generated it.

  • Discoverable means it is possible for researchers to uncover this. A list of desired possibilities, which occurs entirely in the mind and doesn’t consistently direct action, is irrelevant to Rational Choice Theory.
  • Complete means it contains all possible actions and choices. Some of these may be unknown at the time that a decision is made, but once it is known, it does not change the order of preferences.
  • Transitive means the order is consistent, that there are no loops or self-referential cycles. For instance, if you would rather have money than a job, and would rather be comfortable than have money, therefore you would rather be comfortable than have a job.
  • Preference Schedule means that this is the list that controls actions. It’s important to note that Rational Choice Theory is not a psychological theory. There is no need, whatsoever, for Rational Choice to explain the “reasons” for choices, or the subjective experience of the chooser.

It is the transitive requirement which prevents Rational Choice Theory from being a scientific theory. For instance, in the example above, even if we could discover that the subject who prefers a job to comfort, then the Rational Choice Theorist would say there must really be some other elements we hadn’t considered — say a desire to be useful and a desire not to be worthless, which are the real preferences.

Rational Choice Theory is the No True Scotsman fallacy writ large.

All that said, Rational Choice is a method for generating theories. Some are falsified. Others are not and are found to be useful. Like Evolutionary Psychology with its mythical “Era of Evolutionary Adaptation,” Rational Choice’s discoverable complete transitive preference schedule is a tool that enables scientists to create scientific theories about the world, rather than a scientific theory in itself.

Dr. Jackson’s attack on Rational Choice Theory was anti-science, because it privileged his idiosyncratic idealistic prejudices against the scientific method.

He would have been far more useful if he had merely stated it was not a scientific theory at all.