Toward a Unified Account of Finding the Truth: Quine and Bayes on Dogma Versus Good Authority

Scott Alexander recently posted a great account (and critique) of reasoning and communication, beginning with criticism of science communication. He then expands this into an argument about how a real Bayesian evaluation of truth claims requires us to sometimes reject "higher standards" of evidence that produce results in conflict with what we think we already know. In so doing he begins to unify every day reasoning within a Bayesian framework. This is valuable, because honest rationalists have noticed gaps between what we consider high-quality evidence, and what we actually use to update - not because we are hypocrites, but because there are other considerations when we want to really get the right answer.

To begin with a critique of simplistic "levels of evidence": even those of us most enthusiastic about peer-review make almost every decision in life without it. You don't insist on a peer-reviewed journal when figuring out how to give yourself administrator rights to a PC you just got for free (a situation I just found myself in.) Your decision process is a combination of evaluations of the cost, time, and likely consequences of whatever you do, along with deciding what sources to trust based on the decisions' specialization beyond daily experience of subject matter, speed of feedback, and possible perverse incentives. In this case, I just watched a Youtube video and it worked like a charm. It's a free computer I'm playing with so I didn't care much if it was ruined by the attempt.

Taking all these things into account is actually Bayesian even if we aren't thinking explicitly about the Bayes equation. But it turns out the model we use in science is actually a special case of Bayesian reasoning - and even victims of dogmatism are using Bayesian reasoning. The second statement is much more controversial than the first; scroll down to that section if you like.


Recasting Popperian Falsification in Bayesian Terms

Karl Popper's model of hypothesis testing is that we can only falsify a hypothesis, we can never be sure it's true. Looking at Anscombe' quartet above, you can see that there are multiple data sets which can produce the same statistics. Stopping after inadequate positive predictions may lead you to the wrong equation. Therefore, the only answer that provides certainty is to falsify a hypothesis.

There are two important aspects to this approach to finding out the truth. The first, and the more underemphasized, is its appreciation of human psychology. Stephen Toulmin noted that the way humans actually reason is to start with premises and conclusion first, and then build a rhetorical bridge between them. If you're a rationalist, you make sure that your rhetorical bridge is not just verbal/emotional sleight of hand but rather an actual argument. The point is that even in Popperian science, we start with a conclusion. The difference is that we explicitly declare, in public, ahead of time, that we're not sure if the conclusion is right (it's less than a thesis - a "hypo" thesis, if you will.) Then you test it.

And, the way you test it is by creating an experience that will give you an unexpected result if it's wrong - to falsify it. The question is, does a hypothesis-supporting experiment (that produces the results we expect) really give us zero information? No, but it usually doesn't give us as much information as a falsification.

A falsification is usually more strongly weighted information - it "moves the needle" much more, because it's more likely to be surprising. Indeed to equate heavier weighting with surprise is almost tautology. So we can place Popperian hypothesis-testing in a Bayesian framework by saying, instead of a false binary of "support equals zero information equals bad" and falsification good", note that ideally an experiment is trying to create that experience which is most likely to produce a surprise, and most efficiently provide information.


Finding Truth in the Real World - Where Other Humans Also Live

But people don't always do this. We live in a complicated world and have had to learn to weight truth claims based on something besides immediate experience. You almost certainly have not done an experiment on biological evolution, and few if any true retrodictive studies.[1] You can't function otherwise, unless you live in a cabin in the wilderness by yourself.

One thing we use in the real world is a web of beliefs, in the Quinean sense. Recasting this in Bayesian terms, no belief is an island, with each belief in the web serving as part of the weighting for your prior. Most beliefs will have many other beliefs bearing on it, updating the belief in question and holding its prediction in place. If you have reasonable confidence in those many other beliefs, then a counterclaim or weird observation just doesn't outweight them. (I am not a statistician, but Elo ratings seem analagous to this. Just because the Packers had an off night and lost to the Detroit Lions doesn't mean you should bet against the Packers going forward - move your needle? Yes, but this is what it means for one loss to be a fluke.)[2] This is why in the linked article, Scott describes how it is right to reject peer-reviewed homeopathy studies - because they are on their face improbable, in large part because of our rich web of other beliefs about how the world works.

Another way we weight beliefs is by giving credence to authority. I think differing confidence in authorities (based partly on our innate cognitive/emotional makeup and partly on life experience) explains the majority of major differences people have in beliefs about the world - not their direct experience of things or reasoning ability. The truth is that people often say "I believe X because an authority I trust said so." In fact, fellow rationalist, you often say that. And again, to function, you must. (Education is a formal example of this. You don't need to do every experiment back to Newton to be a physicist.) Yet during the pandemic, it has become painfully clear that people differ in what authorities they listen to.


Why Dogmatists Are Actually Still Bayesians: Good Authorities and Bad Authorities

I once had the privilege to give a talk to the Sacramento Freethinkers, Atheists and Nonbelievers (SacFAN) about the function of beliefs. At one point I stated that to have good beliefs, you had to pick good authorities. An audience member asked me "How do you define a good authority?" At the time I answered humorously, deflecting the question only because of my discomfort at realizing I didn't have a ready answer.

First let's define authority: a personal source of data (a person or institution) whose truth claims you weight more heavily than others' without first requiring evidence.

We all have our authorities. To function, we must. Sometimes they are formal (academic scientists); more often they are informal (someone you hike with who seems to know the trails and conditions in your part of the world.)

But there are many claimed authorities with poor justification for their beliefs, that promulgate false beliefs, and if we are updating based on what they tell us, we will be wrong too. A current near-canonical example for my likely readers would be a religious leader claiming that mRNA vaccines will kill you. But scientists can be wrong too, and not just because they haven't generated enough data to update their beliefs. Nobel Prize winners going off the rails are something of an institution now, so how do we know to ignore Kary Mullis or Linus Pauling's weirder moments? Not so scary, you say: partly the conflict with the web of beliefs as noted before, and partly because they're speculating outside of their expertise or reproducible experiment. Fair enough.

Returning to the pandemic, the CDC said for at least first month that masks didn't help. (Was this deliberate obfuscation to keep non-medical people from hoarding masks, or an error? Either way, that's what they communicated, and in any event I'm not sure which is worse.) If you still think, despite this, that the CDC is still a better authority than a preacher, why?


Is Someone Who Calls Himself a Rationalist Claiming Dogmatists are Rational?

In an article about people in middle America who refused the vaccine and got severe COVID, I was struck by the following statement (paraphrasing since I can't find the original): "We didn't think it was real, just like everybody else."

When you are surrounded by people who believe X, all of whom (along with you) admire a leader who believes X, and only people who everyone around you has told you are liars are telling you not-X - then, in the absence of (thusfar) immediately contradictory experience, you will continue to believe X. This is the case for COVID victims like those I paraphrased. Given the information they have, they were being good Quineans and good Bayesians.


Early Life Experience and Emotional Makeup Clearly Influence Our Weightings

You might be asking incredulously: is this guy seriously arguing that people following bad authorities are good Bayesians? And (assuming that's true), does that mean people are really in an inescapable hole if they follow the wrong authority, absent any profound contradictory evidence? The respective answers are technically yes, and sometimes no.

To the first point, you might object that many of these people certainly did have experts providing them better information, and they incorrectly underweighted this information, so they were NOT good Bayesians. But there's a problem: large inferential distance. Everyone around them has told them that (Fauci, academics, etc.) are bad, lying, immoral people who are trying to harm you, and should be ignored. With this information, and very little information about how to identity good authorities, from their perspective, if they give credence to Fauci, they have no justification for not also listening to every crank who comes along. Similarly, it's hard to see how someone in North Korea should somehow just know that people in the West really don't have it out for them. These people do not have trapped priors in the sense that that a belief is somehow innately more inert - as if it has a higher specific epistemological gravity - but their priors are pinned down by a dense web of beliefs whose strands are numerous and have strong weightings because they date to early life experience.

To the second question about whether it's hopeless to get people with bad authorities out of their delusion box, as often happens with the rationalist community, in our public discussions we're playing catch-up centuries after salesmen, politicians and religious leaders figured these things out, though in fairness, they had a greater incentive with more immediate feedback loops. The trick is to find someone who they recognize as an authority. Usually this is as simple as finding people providing better information who share with their audience superficial markers of cultural affinity - afiliations with religions or regions, class, dress, language. Yes these people should understand how to select an authority, but the inferential distance is too great. Put them in touch with someone with the same accent instead. Concentrate forces by looking for people who do not demonstrate "authority monoculture." You can also decrease inferential distance by engaging with people who are already having doubts. (Again, not a news flash, but it may help rationalists to understand if I put it in these terms. See what I did there?)


How to Differentiate Dogmatists from Good Authorities

Let's define dogmatists (or charlatans, or whatever other term you like) as someone who wants the benefits of being an authority, but is not interested in the the truth of the beliefs they promulgate, in terms of actions and consequences. Whether or not they genuinely believe they are interested in the truth is irrelevant, and either way, they will certainly claim they are committed to the truth. There's an analogy here to the relationship between tribal team cheers (shibboleths) that appear on the surface to be truth claims, and proper truth claims, like dolphins and fish. Similarly, dogmatists do their best to masquerade[3] as good authorities, but they're really something else entirely - dogmatists might be considered the viceroys to the good authorities' monarchs.

But there are characteristics of good authorities which are "expensive" for dogmatists to maintain. These are:

1. They have some feedback loop betwen their claims and outcomes (and are interested in it.) Example: physicians and patient outcomes; politicians and legislative impact.
2. They do not avoid being tracked by others in their outcomes or predictions.
3. They minimally appeal to or rely on those early-life, emotional, identity-overweighted beliefs in their audience's web.
4. Their feedback loop is not distorted by perverse interests. Example: TV news pundits trying to get ratings by avoiding predictions that conflict with their audience's values. Paraphrasing Charlie Munger, when you're dealing with a business, you have to understand their business well enough to know their incentives. (More here.)
5. They have a way of seeking out "legitimate surprise" (not mere confusion - can also be accomplished with dopamine hacking, i.e. meth.) Example, hypothesis testing in science. (This may be the hardest of these to do consistently due to inherent contradictions within any information-seeking entity.)
6. They communicate clearly, and they do not hold up incomprehensibility as a positive. (See: descriptions of John von Neumann, the first of Asher's Seven Sins of Medicine.) Insistence of formal or especially foreign or arcane language is one pernicious form. (See: modern legal language, use of Latin in Church, legal French in late Medieval England.)
7. Their claims to authority are limited in scope. Henry Ford had some great industral ideas but also thought people should listen to his hang-ups about cows and horses. Einstein was offered the presidency of Israel but recognizing that his domain of expertise did not extend to politics he said no.

Seeking good authorities, and preparing to reject one you may have liked when you realize they are just a viceroy, is uncomfortable - it's "software" imposed on the factory settings of humans as we operated in small groups for millennia. People who are constitutionally high in the moral dimensions of loyalty and authority will always find this difficult - the idea of checking their proposed authority runs counter to their nature.


For further reading: more formal syntheses of Quinean and Bayesian models here and here. A useful discussion of a possible conflict here, which the resolution that Bayesian reasoning can be a satisfactory way to contruct a Quinean web without arguing that Bayes is necessarily optimal.


FOOTNOTES

[1] Retrodiction sometimes upsets Popperians, but hypothesis-testing is about making a prediction, based on your hypothesis, about something that the predictor does not yet know, even if it already happened. It is always about the state of knowledge of the claimant when the hypothesis is formulated; it's irrelevant if the events already happened, just that they are not yet known.

[2] The ELO rating used in sports not formally an example of a belief web but it is analogous to one, and in this case behaves similar to one. Just as a single peer-reviewed homeopathy study should not make us throw away the rest of science, a great team somehow losing to a bad one should not make us think the great team is now worse than the bad one.

[3] It is a testament to the success of science that viceroys, especially religious ones in the U.S., increasingly co-opt its language. It is very rare to see this happening in the opposite direction.

Business Using Government to Stifle Competition

A smartphone app in New York that would let people find and hail cabs is being profoundly retarded by the city. No doubt such an app would be of great benefit to people hailing cabs and to individual cab drivers, but a disaster for cab companies (and dispatchers). Is there anything more to this story than cab companies protecting their interest at the expense of citizens, and the the city eagerly using regulation to do this for them?

The established businesses are very good at having to avoid defending their behavior; no matter what they say their motivation is, it's pretty obvious that they're acting exactly the same as they would be acting if they were just defending their interests at all costs. The state liquor control board in Pennsylvania has been doing much the same thing.

Regulation is often the friend of big business.

Ridiculous Moral Arguments in Business

Sometimes, a business with a government-backed monopoly on an activity thought to be morally dubious will make funny arguments against the start-up of similar businesses. Here you can read about American states with state-controlled liquor distribution, and the state stores saying as vaguely as possible that we shouldn't have more liquor stores, because they're evil.

The same silliness occurs with casinos and gambling, and especially lotteries. (This particular story reminds me of the hypocrisy around injuries in high school sports, vs. other high school activities). If we're going to rule that there are goods and services too dangerous for the free market, or dangerous enough that they should only exist as government-backed monopolies, we should have a transparent, automated rule for determining which goods and services those are. Otherwise, these decisions will always be made by exactly those parties that have conflicts of interest, and shouldn't be involved in the process.

How Do We Decide When a Good or Service Is Too Dangerous For a Free Market?

There are areas of commerce where markets can be perceived as maintaining a tension with moral values. These areas include but are not limited to: commonly, gambling, prostitution, mind-altering drugs; more centrally in the study of economics, labor arrangements (here and here), and of course health care. We could extend this last category to the laws growing up around the U.S. and elsewhere restricting sale of trans fats or sugary drinks to kids. It's harder to argue (but not impossible) that markets should not be allowed to run their course in things like (for example) clothing. It's surprisingly difficult to think of examples of things that governments have tried to specially insulate from market economics short of 100% centrally planned economies. (In fact if you're worried about the free markets of shoes destroying the world, Douglas Adams anticipated your concern.) What the commonly insulated goods and services have in common is that they're things about which humans have difficulty thinking rationally (short term sensual goods, "vices" - gambling, prostitution, sugary drinks) or no rational basis for their behavior in the first place (avoidance of negatives, like indignity, suffering, and death, i.e. labor arrangements and health care). As a first approximation, the more boring a good or service - i.e., the more that commerce decisions take place entirely in the executive centers of the brain - the fewer problems it will cause on an open market. Shoes are not as dangerous as opiates, or as contentious as indentured servitude.

But humans are not always the rational optimizers we think we are, and it will always be the case that some of us make bad decisions with regards to certain goods or services, and this tendency differs between individuals. Whether it is moral to restrict others from these goods and services is an important question that libertarians wish more people would consider. Case in point: I have given up trying to eat only reasonable, non-health-damaging amounts of chocolate. Is it moral to demand limits, or an outright ban, on chocolate sales for everyone, to protect the weak-willed like myself? Why then for alcohol? Why for prostitution and heroin and gambling? After all, you're mature enough to handle (or just plain avoid) these things, so how can I demand you alter your actions, even though keeping chocolate available absolutely has an impact on my health? So why isn't allowing the poor judgment of some to make the decision for all a classic case of tyranny of the majority?

What's interesting is that as freedoms are extended, the leading narrative-generators in these sorts of debates become less moral and more economic, not to mention more transparently self-serving. That is, at the start of the discussion, we can't make X legal because the world will end, and think about the children! Once X is legal in some polities, it becomes increasingly clear that the world has not ended and the children are fine, so the debate moves to economics. Eventually, no one can remember what the fuss was. Charging interest on loans was considered immoral and an affront to God in European Christianity. How much of that talk have we heard in the Greek crisis so far? Of course, debt can be dangerous, which is exactly what the last few years of economics have shown, but without it, most of us would never own property.

What prompted me to write this post is that, with liberalizing goods (as opposed to de-liberalizing ones like health care in the U.S.) it's during the transition period from world-ending to pure-economics that we hear the funniest arguments. Economic stake-holders want to defend their turf, and their competition is largely held at bay through government-enforced monopolies or oligopolies. At the same time, these stakeholders know that "if you allow more businesses to buy/do X the world will end" is more compelling to a liberal-democratic public than "if you allow more businesses to buy/do X, I'll have more competition", especially if the earlier values that made the good or service a vice persist over time. So a stake holder will pony up to the mic and with a straight face, s/he will tell the public that what s/he does for a living is bad, and there shouldn't be any more allowed. To use two provincial Eastern U.S. examples: "'We have to fight this explosion of gambling all around us,' said Don Marrandino, eastern regional president of Harrah's Entertainment Inc., which has four casinos in Atlantic City." (From "East Coast Casino Market Getting Crowded"; another provincial East Coast issue is the liberalization of alcohol laws in Pennsylvania, allowing sales in supermarkets like most of the U.S. allows, which also gets its share of incoherent moral and economic arguments from, you guessed it, managers of state-run alcohol stores. [Added later: at National Review Reihan Salam analyzes arguments against the spread of online education as being one example of the general phenomenon of traditional producers' inconsistent rhetoric against a shift in production; I would argue the same applies to distribution: "Call it the producer’s lament. Basically, traditional producers — of handicrafts, of pop albums, of community college education — grow accustomed to making a product in a particular way. When cheaper alternatives threaten to put them out of business, we hear a wide array of arguments, many of them conflicting and contradictory (but hey, whatever works!), about why the new mode of production is profoundly dangerous or unjust. Clay Shirky has vividly described how this discourse has played out in the book-selling and publishing industries, as has the always excellent Tim Lee."]
[Added still later - it works in politics too, as in this case where the existing Cherokee Nation is suing Tennessee over its recent recognition of other tribes. Native solidarity, right?]

Of course, there's really no surprise here; people will act in ways that are in their material self-interest, and industry-based rhetoric is no exception. But it's a reminder that capitalists are particularly, as we should expect, materially self-interested - they want wealth, not free markets - and they're willing to use government to that end. Hence we've had admonitions right from the start from the likes of Adam Smith to save capitalism from the capitalists: "The proposal of any new law or regulation of commerce which comes from [businesspeople], ought always to be listened to with great precaution, and ought never to be adopted till after having been long and carefully examined, not only with the most scrupulous, but with the most suspicious attention."

Note the absence of explicit "the world will end" alarmism from the casino manager's rhetoric, because that would require a little bit too much doublethink even for the average news-consumer aiming to reconfirm his or her values. I'm looking forward to seeing the same kinds of self-oblivious contortions from money-lending banks in majority-Islamic countries this century, as well as from marijuana sellers (or local governments benefiting from tax revenues) in parts of the U.S. that are liberalizing marijuana laws: here, here and here.