Posts Tagged Behavioral Economics

Intuitive Bayes and the preponderance of pessimism about human reasoning

(This is a 4th post on rational behavior of people unfairly judged to be irrational. See 1, 2, and 3)

Most people believe they are better than average drivers. Is this a cognitive bias? Behavioral economists think so: “illusory superiority.” But a rational 40-year-old having had no traffic accidents might think her car insurance premiums are still extremely high. She may then conclude she is a better than average drivers since she’s apparently paying for a lot of other peoples’ smashups. Are illusory superiority and selective recruitment at work here? Or is this intuitive Bayesianism operating on the available evidence?

Bayesian philosophy is based on using a specific rule set for updating one’s belief in light of new evidence. Objective Bayesianism, in particular, if applied strictly, would require us to quantify every belief we hold – our prior credence –  with a probability in the range of zero to one and to quantify the value of new evidence. That’s a lot of cognizing, which would lead to a lot more personal book keeping than most of us care to do.

As I mentioned last time, Daniel Kahneman and other in his field hold that we are terrible intuitive Bayesians. That is, they believe we’re not very good at doing the equivalent of Bayesian reasoning intuitively (“not Bayesian at all” said Kahneman and Tversky in Subjective probability: A judgment of representativeness, 1972). But beyond the current wave of books and TED talks framing humans as sacks of cognitive bias (often with government-paternalistic overtones), many experts in social psychology have reached the opposite conclusion.

For example,

  • Edwards, W. 1968. “Conservatism in human information processing”. In Formal representation of human judgment.
  • Peterson, C. R. and L. R. Beach. 1967. “Man as an intuitive statistician”. Psychological Bulletin. 68.
  • Piaget, Jean. 1975. The Origin of the Idea of Chance in Children.
  • Anderson, J. R. 1990. The Adaptive Character of Thought.

Anderson makes a particularly interesting point. People often have reasonable but wrong understandings of base rates, and official data sources often vary wildly about some base rates. So what is characterized by critics of humans’ poor performance at Bayesian reasoning (e.g., by ignoring rates) is in fact use of incorrect base rates, not a failure to employ base rates at all.

Beyond the simple example above of better-than-average driver belief, many examples have been given (and ignored by those who see bias everywhere) of intuitive Bayesian reasoning that yields rational but incorrect results. These include not only for single judgments, but for people’s modification of belief across time – Bayesian updates.

For math-inclined folk seeking less trivial examples, papers like this one from Benoit and Dubra lay this out in detail (If a fraction x of the population believes that they rank in, say, the top half of the distribution with probability at least q > 1/2, then Bayesian rationality immediately implies that xq <= 1/2, not that x <= 1/2 [where q is the subject’s confidence that he is in the top half and x is the fraction who think they’re in the top half]).

A 2006 paper, Optimal Predictions in Everyday Cognition, by Thomas L. Griffiths and Joshua B. Tenenbaum warrants special attention. It is the best executed study I’ve ever seen in this field, and its findings are astounding – in a good way. They asked subjects to predict the duration or extent of common phenomena such as human lifespans, movie run times, and the box office gross of movies. They then compared the predictions given by participants with calculations from an optimal Bayesian model. They found that, as long as subjects had some everyday experience with the phenomena being predicted (like box office gross, unlike the reign times of Egyptian pharaohs), people predict extremely well.

The results of Griffiths and Tenenbaum showed people to be very competent intuitive Bayesians. Even more interesting, people’s implicit beliefs about data distributions, be they Gaussian (birth weights), Erlang, (call-center hold times), or power-law (length of poems), were very consistent with real works statistics, as was hinted at in Adaptive Character of Thought.

Looking at the popular material judging people to be lousy Bayesians steeped in bias and systematic error, and far less popular material like that from Griffiths/Tenenbaum, Benoit/Dubra and Anderson, makes me think several phenomena are occurring. To start, as noted in previous posts, those dedicated to uncovering bias (e.g. Kahneman, Ariely) strongly prefer confirming evidence over disconfirming evidence. This bias bias manifests itself both as ignoring cases where humans are good Bayesians reaching right conclusions (as in Griffiths/Tenebaum and Anderson) and as failure to grant that wrong conclusions don’t necessarily mean bad reasoning (auto driver example and the Benoit/Dubra cases).

Further, the pop-science presentation of human bias (Ariely TED talks, e.g.) makes newcomers to the topic feel like they’ve received a privileged view into secret knowledge. This gives the bias meme much stronger legs than the idea that humans are actually amazingly good intuitive Bayesians in most cases. As John Stuart Mill noted 200 years ago, those who despair when others hope are admired as sages while optimists are dismissed as fools. The best, most rigorous analyses in this realm, however, rest strongly with the optimists.

 

1 Comment

Daniel Kahneman’s Bias Bias

(3rd post on rational behavior of people too hastily judged irrational. See first and second.)

Daniel Kahneman has made great efforts to move psychology in the direction of science, particularly with his pleas for attention to replicability after research fraud around the priming effect came to light. Yet in Thinking Fast And Slow Kahneman still seems to draw some broad conclusions from a thin mantle of evidentiary icing upon a thick core of pre-formed theory. He concludes that people are bad intuitive Bayesians through flawed methodology and hypotheticals that set things up so that his psychology experiment subjects can’t win. Like many in the field of behavioral economics, he’s inclined to find bias and irrational behavior in situations better explained by the the subjects’ simply lacking complete information.

Like Richard Thaler and Dan Ariely, Kahneman sees bias as something deeply ingrained and hard-coded, programming that cannot be unlearned. He associates most innate bias with what he calls System 1, our intuitive, fast thinking selves. When called on to judge probability,” Kahneman says, “people actually judge something else and believe they have judged probability.” He agrees with Thaler, who finds “our ability to de-bias people is quite limited.”

But who is the “we” (“our” in that quote), and how is that “they” (Thaler, Ariely and Kahneman) are sufficiently unbiased to make this judgment? Are those born without the bias gene somehow drawn to the field of psychology; or through shear will can a few souls break free? If behavioral economists somehow clawed their way out of the pit of bias, can they not throw down a rope for the rest of us?

Take Kahneman’s example of the theater tickets. He compares two situations:

A. A woman has bought two $80 tickets to the theater. When she arrives at the theater, she opens her wallet and discovers that the tickets are missing. $80 tickets are still available at the box office. Will she buy two more tickets to see the play?

B. A woman goes to the theater, intending to buy two tickets that cost $80 each. She arrives at the theater, opens her wallet, and discovers to her dismay that the $160 with which she was going to make the purchase is missing. $80 tickets are still available at the box office. She has a credit card. Will she buy the tickets and just charge them?

Kahnemen says that the sunk-cost fallacy, a mental-accounting fallacy, and the framing effect account for the fact that many people view these two situations differently. Cases A and B are functionally equivalent, Kahneman says.

Really? Finding that $160 is missing from a wallet would cause most people to say, “darn, where did I misplace that money?”. Surely, no pickpocket removed the cash and stealthily returned the wallet to her purse. So the cash is unarguably a sunk cost in case A, but reasonable doubt exists in case B. She probably left the cash at home. As with philosophy, many problems in psychology boil down to semantics. And like the trolley problem variants, the artificiality of the problem statement is a key factor in the perceived irrationality of subjects’ responses.

By framing effect, Kahneman means that people’s choices are influenced by whether two options are presented with positive or negative connotations. Why is this bias? The subject has assumed that some level of information is embedded in the framer’s problem statement. If the psychologist judges that the subject has given this information too much weight, we might consider demystifying the framing effect by rebranding it the gullibility effect. But at that point it makes sense to question whether framing, in a broader sense, is at work in the thought problems. In presenting such problems and hypothetical situations to subjects, the framers imply a degree of credibility that is then used against those subjects by judging them irrational for accepting the conditions stipulated in the problem statement.

Bayesian philosophy is based on the idea of using a specific rule set for updating a “prior” (meaning prior belief – the degree of credence assigned to a claim or proposition) on the basis of new evidence. A Bayesian would interpret the framing effect, and related biases Kahneman calls anchoring and priming, as either a logic error in processing the new evidence or as a judgment error in the formation of an initial prior. The latter – how we establish initial priors – is probably the most enduring criticism of Bayesian reasoning. More on that issue later, but a Bayesian would say that Kayneman’s subjects need training in the use of uninformative priors and initial priors. Humans are shown to be very trainable in this matter, against the behavioral economists’ conclusion that we are hopelessly bound to innate bias.

One example Kahneman uses to show the framing effect presents different anchors to two separate test groups:

Group 1: Is the height of the tallest redwood more or less than 1200 feet? What is your best guess for the height of the tallest redwood?

Group 2: Is the height of the tallest redwood more or less than 120 feet? What is your best guess for the height of the tallest redwood?

Group 1’s average estimate was 844 feet, Group 2 gave 282 feet. The difference between the two anchors is 1080 feet. (1200 – 120). The difference in estimates by the two groups was 562 feet. Kahneman defines anchoring index as the ratio of the difference between mean estimates and difference in anchors. He uses this anchoring index to measure the robustness of the effect. He rules out the possibility that anchors are taken by subjects to be informative, saying that obviously random anchors can be just as effective, citing a 50% anchoring index when German judges rolled loaded dice (allowing only values of 3 or 9 to come up) before sentencing a shoplifter (hypothetical, of course). Kahneman reports that judges rolling a 3 gave 5-month sentences while those rolling a 9 assigned the shoplifter an 8-month sentence (index = 50%).

But the actual study (Englich, et. al.) cited by Kahneman has some curious aspects, besides the fact that it was very hypothetical. The judges found the fictional case briefs to be realistic, but they were not judging from the bench. They were working a thought problem. Englich’s Study 3 (the one Kahneman cites) shows the standard deviation in sentences was relatively large compared to the difference between sentences assigned by the two groups. More curious is a comparison of Englich’s Study 2 and the Study 3 Kahneman describes in Fast and Slow. Study 2 did not involve throwing dice to create an anchor. Its participants were only told that the prosecutor was demanding either a 3 or 9 month sentence, those terms not having originated in any judicial expertise. In Study 3, the difference between mean sentences from judges who received the two anchors was only two months (anchoring index = 33%).

Studies 2 and 3 therefore showed a 51% higher anchoring index for an explicitly random (clearly known to be random by participants) anchor than for an anchor understood by participants to be minimally informative. This suggests either that subjects regard pure chance as being more useful than potentially relevant information, or that something is wrong with the experiment, or that something is wrong with Kahnemnan’s inferences from evidence. I’ll suggest that the last two are at work, and that Kahneman fails to see that he is preferentially selecting confirming evidence over disconfirming evidence because he assumed his model of innate human bias was true before he examined the evidence. That implies a much older, more basic fallacy might be at work: begging the question, where an argument’s premise assumes the truth of the conclusion.

That fallacy is not an innate bias, however. It’s a rhetorical sin that goes way back. It is eminently curable. Aristotle wrote of it often and committed it slightly less often. The sciences quickly began to learn the antidote – sometimes called the scientific method – during the Enlightenment. Well, some quicker than others.

,

Leave a comment