In defence of epistemic modesty

This piece defends a strong form of epistemic modesty: that, in most cases, one should pay scarcely any attention to what you find the most persuasive view on an issue, hewing instead to an idealised consensus of experts. I start by better pinning down exactly what is meant by ‘epistemic modesty’, go on to offer a variety of reasons that motivate it, and reply to some common objections. Along the way, I show common traps people being inappropriately modest fall into. I conclude that modesty is a superior epistemic strategy, and ought to be more widely used – particularly in the EA/rationalist communities.

[gdoc]

Provocation

I argue for this:

In virtually all cases, the credence you hold for any given belief should be dominated by the balance of credences held by your epistemic peers and superiors. One’s own convictions should weigh no more heavily in the balance than that of one other epistemic peer.

 

Introductions and clarifications

A favourable motivating case

Suppose your mother thinks she can make some easy money day trading blue-chip stocks, and plans to kick off tomorrow shorting Google on the stock market, as they’re sure it’s headed for a crash. You might want to dissuade her in a variety of ways.

You might appeal to an outside view:

Mum, when you make this short you’re going to be betting against some hedge fund, quant, or whatever else. They have loads of advantages: relevant background, better information, lots of data and computers, and so on. Do you really think you’re odds on to win this bet?

Or appeal to some reference class:

Mum, I’m pretty sure the research says that people trying to day-trade stocks tend not to make much money at all. Although you might hear some big successes on the internet, you don’t hear about everyone else who went bust. So why should you think you are likely to be one of these remarkable successes?

Or just cite disagreement:

Look Mum: Dad, sister, the grandparents and I all think this is a really bad idea. Please don’t do it!

Instead of directly challenging the object level claim (i.e. “Google isn’t overvalued, because X”). These considerations attempt to situate the cogniser within some population, and from characteristics of this population infer the likelihood of this cogniser getting things right.  

Call the practice of using these techniques considerations epistemic modesty. We can distinguish two components:

  1. ‘In theory’ modesty: That considerations of this type should in principle influence our credences.
  2. ‘In practice’ modesty: That one should in fact use these considerations when forming credences.

Weaker and stronger forms of modesty

Some degree of modesty is (almost) inarguable. If one leaves for work on Tuesday and finds all your neighbours left their bins out, that’s at least reason to doubt your belief bins were on Thursday, and perhaps sufficient to believe instead bins are on Tuesday (and follow suit with your bins). If it appears that, say, the coagulation cascade ‘couldn’t evolve’, the near unanimity of assent for evolution among biologists at least counts against this, if not a decisive reason, despite one’s impressions, that it could. Nick Beckstead suggests something like ‘elite common sense’ forms a prior which one should be hesitant to diverge from without good reason. 

I argue for something much stronger (c.f. the Provocation above): in theory, one’s credence in some proposition P should be almost wholly informed by modest considerations. That, ceteris paribus, the fact it appears to you that P should weigh no more heavily in one’s determination regarding P than knowing that it appears to someone else that P. Not only is this the case in theory, but it is also the case in practice. One’s all things considered judgement on P should be just that implied by an idealized expert consensus on P, no matter one’s own convictions regarding P.

Motivations for more modesty

Why believe ‘strong form’ epistemic modesty? I first show families of cases where ‘strong modesty’ leads to predictably better performance, and show these results generalise widely.[1] 

The symmetry case    

Suppose Adam and Beatrice are perfect epistemic peers, equal in all respects which could bear on them forming more or less accurate beliefs. They disagree on a particular proposition P (say “This tree is an Oak tree”). They argue about this at length, such that all considerations Adam takes to favour “This is an Oak tree” are known to Beatrice, and vice versa.[2] After this, they still disagree: Adam has a credence of 0.8, Beatrice 0.4.

Suppose an outside party (call him Oliver) is asked for his credence of P, given Adam and Beatrice’s credences and their epistemic peer-hood to one another, but bereft of any object-level knowledge. He should split the difference between Adam and Beatrice – 0.6: Oliver doesn’t have any reason to favour Adam over Beatrice’s credence for P as they are epistemic peers, and so splitting the difference gives the least expected error.[3] If he was faced with a large class of similar situations (maybe Adam and Beatrice get into the same argument for Tree 2 to Tree 10,000) Oliver would find that difference splitting has lower error than biasing to either Adam or Beatrice’s credence.

Adam and Beatrice should do likewise. They also know they are epistemic peers, and so they should also know that for whatever considerations explain their difference (perhaps Adam is really persuaded by the leaf shapes, but Beatrice isn’t) Adam’s take and Beatrice’s take are no more likely to be right than one another. So Adam should go (and Beatrice vice-versa), “I don’t understand why Beatrice isn’t persuaded by the leaf shapes, but she expresses the same about why I find it so convincing. Given she is my epistemic peer, ‘She’s not getting it’, and, ‘I’m not getting it’ are equally likely. So we should meet in the middle”.

The underlying intuition is one of symmetry. Adam and Beatrice have the same information. The correct credence regarding P given this information should not depend on which brain Adam or Beatrice happens to inhabit. Given this, they should hold the same credence[4], and as they Adam is as likely to be further from the truth than Beatrice, the shared credence should be in the middle.  

Compressed sensing of (and not double-counting) the object level

It seems odd that both Adam and Beatrice do better discarding their object level considerations regarding P. If we adjust the scenario above so they cannot discuss with one another but are merely informed of each other’s credences (and that they are peers regarding P), the right strategy remains to meet in the middle.[5] Yet how come Adam and Beatrice are doing better if they ignore relevant information? Both Adam and Beatrice have their ‘inside view’ evidence (i.e. what they take to bear on the credence of P) and the ‘outside view’ evidence (what each other think about P). Why not use a hybrid strategy which uses both?

Yet to whatever extent Adam or Beatrice’s hybrid approach leads them to diverge from equal weight, they will do worse. Oliver can use the ‘meet in the middle strategy’ to get an expectedly better accuracy than either biasing towards their own inside view determination. In betting terms, Oliver can arbitrage any difference in credence between Adam and Beatrice.

We can explain why: the credences Adam and Beatrice offer can be thought of as very compressed summaries of the considerations they take to bear upon P. Whatever ‘inside view’ considerations Adam took to bear upon P are already ‘priced in’ to the credence he reports (ditto Beatrice). Modesty is not ignoring this evidence, but weighing it appropriately: if Adam then tries to adjust the outside view determination by his own take on the balance of evidence, he double counts his inside view: once in itself, and once more by including his credence as weighing equally to Beatrice’s in giving the outside view.

One’s take on the set of considerations regarding P may err, either by bias,[6] ignorance, or ‘innocent’ mistake. Splitting the difference between you and your peer’s very high level summary of these captures the great fraction of benefit of hashing out where these summaries differ.[7] Modesty correctly diagnoses that one’s high level summary is no more likely to be more accurate than one’s peers, and so holds those in equal regard, even in cases where the components of one’s own summary are known better.

Repeated measures, brains as credence censors, and the wisdom of crowds

Modesty outperforms non-modesty in the n=2 case. The degree of outperformance grows (albeit concavely) as n increases.

Scientific fields often have to deal with unreliable measurement. They commonly mitigate this by having repeat measurement. If you have a crummy thermometer, repeating readings several times improves accuracy over just the once. Human brains also try and measure things, and they are also often unreliable. It is commonly observed that nonetheless the average of their measurement tends to lie closer to the mark than the vast majority of individual measurements. Consider the commonplace ‘guess how many skittles are in this jar’ or similar estimation games: the usual observation is that the average of all the guesses is better than all (or almost all) the individual guesses.

A toy model makes this unsurprising. The individual guesses will form some distribution centered on the true value. Thus the expected error of a given individual guess is the standard deviation of this distribution. The expected error of the average of all guesses is given by the standard error, which is the standard deviation divided by root(number of guesses):[8] with 10 individuals, the error is about 3 times smaller than the expected error of each individual guess; with 100, 10 times smaller; and so on.

Analogously, human brains also try to measure credences or degrees of belief, and are similarly imperfect to when they’re trying to estimate ‘number of X’. Yet one may expect a similar effect to this ‘wisdom of crowds’ to operate here too. In the same way Adam and Beatrice would do better in the situation above if they took the average (even if it went against their view of the balance of reasons by their lights), if Adam-to-Zabaleta (all epistemic peers) investigated the same P, they’d expect to do better if they took the average of their group versus steadfastly holding to the credence they arrived at ‘by their lights’. Whatever inaccuracies that may throw off their individual estimates of P somewhat cancel out.

Deferring to better brains

The arguments above apply to cases where one is an epistemic peer. If not, one needs to adjust by some measure of ‘epistemic virtue’. In cases where Adam is an epistemic superior to Beatrice, they should meet closer to Adam’s view, commensurate with the degree of epistemic superiority (and vice versa).

Although reasons for being an epistemic superior could be ‘they’re a superforecaster’ or ‘they’re smarter than I am’, perhaps the most common source of epistemic superiors lie under the heading of ‘subject matter expert’. On topics from human nutrition, to voting rules, to the impact of the minimum wage, to the nature of consciousness, to basically anything that isn’t trivial, one can usually find a fairly large group of very smart people who spend many years studying that topic, who make public their views about this topic (sometimes not even behind a paywall). That they at least have a much greater body of relevant information and have spent longer thinking about it gives them a large advantage compared to you.

In such cases, the analogy might be that your brain is a sundial, whilst theirs is an atomic clock. So if you have the option of taking their readings rather than yours, you should do so. The evidence a reading of a sundial provides about the time conditional on the atomic clock reading is effectively zero. ‘Splitting the difference’ in analagous epistemic cases should result with both you and your epistemic superior agreeing that they are right and you are wrong. 

Inference to the ideal epistemic observer

We can summarise these motivations by analogy to ideal observers (used elsewhere in perception and ethical theory). We can gesture that an ideal (epistemic) observer is just that which is able to form the most accurate credence for P given whatever prior: we can explain they have vast intelligence, full knowledge of all matters that bear upon P, perfect judgement, and in essence all epistemic virtues in excelsis.

Now consider this helpful fiction:

The epistemic fall: Imagine a population solely comprised of ideal observers, who all share the same (correct) view on P. Overnight their epistemic virtues are assailed: they lose some of their reasoning capacity; they pick up particular biases that could throw them one way or another; they lose information, and so on, and each one to varying degrees.

They wake up to find they now have all sorts of different credences about P, and none of them can remember what credence they all held yesterday. What should they do?

It seems our fallen ideal observers can begin to piece together what their original credence was about P by finding out more about their credences and remaining epistemic virtue, and so backpropagate their return to epistemic apotheosis. If they find they’re all similarly virtuous and are evenly scattered, their best guess is the ideal observer was in the middle of the distribution (c.f. the wisdom of crowds). If they see a trend that those with greater residual virtue tend to hold a higher credence in P, they should attempt to extrapolate this trend to suggest the ideal agent origin from which they were differentially blown of course from. If they see one group demonstrates a bias that others do not, they can correct the position of this group before trying these procedures. If they find the more virtuous agents are more scattered regarding P, (or that they segregate into widely dispersed aggregations), this should make them very unsure about where the ideal observer initially was. And so on.

Such a model clarifies the benefit of modesty. Although we didn’t have some grand epistemic fall, it is clear we all fall manifestly short of an ideal observer. Yet we all fall short in different respects, and in different degrees. One should want to believe whatever one would believe if one was an ideal observer, shorn of one’s manifest epistemic vices. Purely immodest views must say their best guess is the ideal observer would think the same as they do, and hope that all the vicissitudes of their epistemic vice happen to cancel out. By accounting for the distribution of cognisers, modesty allows a much better forecast, and so a much more accurate belief. And the best such forecast is the strong form of modesty, where one’s particular datapoint, in and of itself, should not be counted higher than any other.

Excursus: Against common justifications for immodesty

So much for strong modesty in theory. How does it perform in practice?

One rough heuristic for strong modesty is this: for any question, find the plausible expert class to answer that question (e.g. if P is whether to raise the minimum wage, talk to economists). If this class converges on a particular answer, believe that answer too. If they do not agree, have little confidence in any answer. Do this no matter whether one’s impression of the object level considerations that recommend (by your lights) a particular answer.

Such a model captures all the common sense cases of modesty – trust the results in typical textbooks, defer to consensus in cases like when to put the bins out, and so on. I now show it is also better in many cases where people think it is better to be immodest.

Being ‘well informed’ (or even true expertise) is not enough

A common refrain is that one is entitled to ‘join issue’ with the experts due to one having made some non-trivial effort at improving one’s knowledge of the subject. “Sure, I accept experts widely disagree on macro-economics, but I’m confident in neo-Keynesianism after many months of careful study and reflection.”

This doesn’t fly by the symmetry argument above. Our outsider observes widespread disagreement in the area of macroeconomics, and that many experts who spend years on the subject nonetheless greatly disagree. Although it is possible the ideal observer would have been in one or another of the ‘camps’ (the clustering implies intermediate positions are less plausible), the outsider cannot adjudicate which one if we grant the economists in each appear to have similar levels of epistemic virtue. The balance of this outside view changes imperceptibly if another person who despite a few months of study remains nowhere near peerhood (let alone superiority) of these divided experts, happens to side with one camp or another. By symmetry, one’s own view of the balance of reason should remain unchanged if this ‘another person’ happened to be you.

The same applies even if you are a bona fide expert. Unless the distribution of expertise is such that there is a lone ‘world authority’ above all others (and you’re them) your fellow experts form your epistemic peer group. Taking the outside view is still the better bet: the consensus of experts tends to be right more often than dissenting experts, and so some difference splitting (weighed more to the consensus owing to their greater numbers) is the right answer.[9]

Common knowledge ‘silver bullet arguments’

Suppose one takes an introductory class in economics. From this, one sees there must be a ‘knock-down’ argument against a minimum wage:

Well, suppose you’re an employee whose true value on the free market is less than the minimum wage. But under the minimum wage, the firm might not decide on charitably employing above your market value, and just firing you instead. You’re worse off, as you’re on the dole, and the firm’s worse off, as it has to meet its labour demand another way. Everyone’s lost! So much for the minimum wage!

Yet one quickly discovers economists seem to be deeply divided over the merits of the minimum wage (as they are about most other things). See for example this poll suggesting 38 economic experts in the US are pretty evenly divided on whether the minimum wage would ‘hit’ employment for low-skill workers, and leant in favour of the minimum wage ‘all things considered’.

It seems risible to suppose these economists don’t know their economics 101. What seems much more likely is that they know other things that you don’t which make the minimum wage more reasonable than your jejune understanding of the subject suggests. One need not belabour which side the outside view strongly prefers.

Yet it is depressingly common for people to confidently hold that view X or Y is decisively refuted by some point or another, notwithstanding the fact this point is well known to the group of experts that nonetheless hold X or Y. Of course in some cases one really has touched on the decisive point the experts have failed to appreciate. More often, one is proclaiming that one is on the wrong side of the Dunning-Kruger effect.

Debunking the expert class (but not you)

To the litany of cases where (apparent) experts screwed up, we can add verses without end. So we might be inclined to debunk a particular ‘expert consensus’ due to some bias or irrationality we can identify. Thus, having seen there are no ‘real’ experts to help us, we must look at the object level case.

The key question is this: “How are you better?” And it is here that debunking attempts often flounder:

An undercutting defeater for one aspect of epistemic superiority for the expert class is not good enough. Maybe one can show the expert class has a poor predictive track record in their field. Unless one has a better track record in their field, this puts you on a par with respect to this desideratum of epistemic virtue. They likely have others (e.g. more relevant object-level knowledge) that should still give them an edge, albeit attenuated.

An undercutting defeater that seems to apply equally well to oneself as the expert class also isn’t enough. Suppose (say) economics is riven by ideological bias: why are you less susceptible to these biases? The same ideological biases that might plague professional economists may also plague amateur economists, but the former retain other advantages.

Even if a proposed debunking is ‘selectively toxic’ to the experts versus you, it still might be your epistemic superior all things considered. Both Big Pharma and Professional Philosophy may be misaligned, but perhaps not so much to be orthogonal or antiparallel to the truth: in both they still expectedly benefit by finding drugs that work or making good arguments respectively. They may still fare better overall than, “Intelligent layperson who’s read extensively”, even if they are not subject to ‘publish or perish’ or similar.

Even if a proposed debunking shows one as decisively superior to that expert class, there may be another expert class which remains epistemically superior to you. Maybe you can persuasively show professional philosophers are so compromised on consciousness that they should not be deferred to about it. Then the real expert class may simply switch to something like ‘intelligent people outside the academy who think a lot about the topic’. If it’s the case that this group of people do not share your confidence in your view, it seems outsiders should still reject it – as should you.

It need not be said that the track record for these debunking defeaters is poor. Most crackpots have a persecution narrative to explain why the mainstream doesn’t recognise or understand them, and some of the most mordant criticisms of the medical establishment arise from those touting complementary medicine. Thus ‘explaining away’ expert disagreement may not put one in a more propitious reference class than one started from. One should be particularly suspicious of debunking(s) sufficiently general that the person holding the unorthodox view has no epistemic peers – they are akin to Moses, descending from Mt. Sinai, bringing down God-breathed truth for the rest of us.[10]

Private evidence and pet arguments

Suppose one thinks one is in receipt of a powerful piece of private evidence: maybe you’ve got new data or a new insight. So even though the experts are generally in the right, in this particular case they are wrong because they are unaware of this new consideration.

New knowledge will not spread instantaneously, and that someone can be ‘ahead of the curve’ comes as no surprise. Yet many people who take themselves to have private evidence are wrong: maybe experts know about it but don’t bother to discuss it because it is so weak, or it is already in the literature (but you haven’t seen it), or it isn’t actually relevant to the topic, or whatever else. Most mavericks who take themselves to have new evidence that overturns consensus are mistaken.

The natural risk is people tend to be too partial to their pet arguments or pet data, and so give them undue weight, and so one’s ‘insider’ perceptions should perhaps be attenuated by this fact. I suspect most are overconfident here.[11] If this private evidence really is powerful, one should expect it to be persuasive to members of this expert class once they become aware of it. So it seems the credence one should have is the (appropriately discounted) forecast of what the expert class would think once you provide them this evidence.

The natural test of the power of this private evidence is to make it public. If one observes experts (or just epistemic peers) shift to your view, you were right about how powerful this evidence was. If instead one sees a much more modest change in opinion, this should lead one to downgrade your estimate as to how powerful this evidence really is (and perhaps provide calibration data for next time). Holding instead this really is decisive evidence leads one to the problematic ‘common knowledge silver bullet’ case discussed above. Inferring from this experts just can’t understand your reasoning or are biased against outsiders or whatever else produces a suspiciously self-serving debunking argument, also discussed above.  

Objections

So much for the case in favour. What about the case against? I divide objections into those ‘in theory’, and those ‘in practice’.

In theory

There’s no pure ‘outside view’[12]

It is not the case you can bootstrap an outside view from nothing. One needs to at least start with some considerations as to what makes one an epistemic peer or superior, and probably some minimal background knowledge of ‘aboutness’ to place topics under one or another expert class. 

In the same way large amounts of our empirical information are now derived by instrument rather than direct application of our senses (but were ultimately germinated from direct sensory experience), large amounts of our epistemic information can be derived by deferring to better (or more) brains rather than using our own, even if this relies on some initial seed epistemology we have to realise for ourselves. This ‘germinal set of claims’ can still be modestly revised later.

Immodestly modest?

One line of attack from the social epistemology literature is that strong forms of modesty are self-defeating. If one is modest, one should assumedly be modest about ‘What is the right way to form beliefs if epistemic peers disagree with you?’ Yet one finds that very few people endorse the sort of epistemic modesty advocated above. When one looks among potential expert classes, such as more intelligent friends of mine (i.e. friends of mine), epistemologists, and so on, conciliatory views like these command only a minority. So the epistemically modest should vanish as they defer to the more steadfast consensus.

If so, so much the worse for modesty. I offer a couple of incomplete defences:

One is haggling over the topic of disagreement. In my limited reading of ‘equal weight/conciliatory views and their detractors’, I take the detractors to be suggesting something like “one is ‘within one’s rights’ to be steadfast”, rather than something like “you’re more accurate if you’re steadfast”. Maybe there are epistemic virtues which aren’t the same as being more accurate. Yet there may be less disagreement on ‘conditional on an accuracy first view, is modesty the right approach?’

This only gets so far (after all, shouldn’t we be modest whether only to care about accuracy?) A more general defence is this: the ‘what if you apply the theory to itself?’ problem looks pretty pervasive across theories.[13] Accounts of moral uncertainty that in whatever sense involve weighing normative theories by their plausibility tend to run into problems if the same accounts are applied ‘one level up’ to meta-moral uncertainty. Bayesian accounts of epistemology seem to go haywire if we think one should have a credence in Bayesian epistemology itself, especially if one assigns any non-zero credence on any theory which entails object level credences have undefined values.

Closer to home, milder versions of conciliation (e.g. “Pay some attention to peer disagreement, but it’s not the only factor”) share a similarly troublesome recursive loop (“Well, I see most other people are steadfast, so I should update to be a bit less conciliatory, but now I have to apply my modified view to this disagreement again”) and neat convergence is not guaranteed. The theories which avoid this problem (e.g. ‘Wholly steadfast, so peer disagreement should be ignored’), tend to be the least plausible on the object level (e.g. That if you believe bins are on Thursday, the fact all your neighbours have their bins out on Tuesday is not even reason to reconsider your belief).

A solution to these types of problems remains elusive. Yet modesty finds itself in fairly good company. It may be the case that a good resolution to this type of issue would rule out the strong form of modesty advocated here, in favour of some intermediate view. Until then, I hope the (admittedly inelegant) “Be modest, save for meta-epistemic norms about modesty itself” is not too great a cost to bear across the scales from the merits of the approach.

In practice

I take most of the action to surround whether modesty makes sense as a practical procedure in the real world, even granting it’s ‘in theory’ virtue. Given the strength of modesty, I advocate, the fact we use something like it in some cases, and we can identify it can help in others, is not enough. It needs to be shown as a better strategy than even slightly weaker forms, in circumstances deliberately selected to pose the greatest challenge to strong modesty.

Trivial (and less trivial) non-use cases

For some topics there’s no relevant epistemic peers or superiors to consider. This is commonly the case with pretty trivial beliefs (e.g. my desk is yellow).

Modesty also doesn’t help much for individual tastes, idiosyncrasies, or circumstances. If Adam works best listening to Bach and Beatrice to Beethoven, they probably won’t do better ‘meeting in the middle’ and both going half-and-half for each (or maybe picking a composer intermediate in history, like Mozart). Anyway, Adam is probably Beatrice’s significant epistemic superior on “What music does Adam work best listening to?”, and vice-versa. One can also be credulous of claims like “It turned out this diet really helped my back pain”: perhaps it’s placebo, or perhaps it is one of those cases where different things work for different people, and one expects in such cases individuals to have privileged access to what worked for them.[14] 

There will be cases where one really is plowing a lonely furrow where there aren’t any close epistemic peers or superiors. It’s possible I really am the world’s leading expert on “How many counter-factual DALYs does a doctor avert during their career?”, because no one else has really looked into this question. My current role involves investigating global catastrophic biological risks, which appears understudied to the point of being pre-paradigmatic.

These comprise a very small minority of topics I have credences about. Yet even here modesty can help. One can use more distant bodies of experts: I am reassured that my autumnal estimate for the ‘DALY question’ coheres with expert consensus that medical practice had a minor role in improvements to human health, for example. Even if I don’t have any epistemic peers, I can simulate some by asking, “If there were lots of people as or more reasonable than me looking at this, would I expect them to agree with my take?” Given that the econometric-esque methods I deploy to the answer the ‘DALY question’ could probably be done better by an expert, and in any case reasonable people are often sceptical of these in other areas, I am less confident of my findings than my ‘inside view’ suggests, which I take to be a welcome corrective to ‘pet argument’ biases.[15]    

In theory, the world should be mad

Whether devoured by Moloch, burned by Ra, trapped by aberrant signalling equilibria, or whatever else, we can expect to predict when apparent expert classes (and apparent epistemic peers) are going to collectively go wrong. With this knowledge, we can know which topics we should expect to ourselves to outperform expertise. Rather than the scenario where we commonly find ourselves looking up (at experts) or around (at our peers), we find ourselves in many situations where those who are usually epistemic peers or superiors are below us – and above us, only sky.

We could distinguish two sorts of madness, a surprising absence of expertise and a surprising error of expertise:

The former is a gap in the epistemic market. Although an important topic should be combed over by a body of experts, for whatever reason it isn’t, and so it takes surprisingly little effort to climb to the summit of epistemic superiority. In such cases our summaries of expert classes as ranging over a broad area conceal the degree of expertise is very patchy: public health experts generally know a great deal about the health impacts of smoking; they usually know much less about the health impacts of nicotine.

The latter is a stronger debunking argument. One appeals to some features of the world that generates expertise and suggests that these expertise generating features are anti-correlated to the truth, thus one can adjudicate between warring expert camps (or just indict all so-called ‘experts’) based on this knowledge. One strong predictor of incompatibilism regarding free will among philosophers is believing in God. If we are confident these beliefs in God are irrational, then we can winnow the expert class by this consideration and side with the compatibilist camp much more strongly.

Yet, similar to the problems of debunking mentioned earlier, that there is a good story suggesting one of these things does not imply one will do better ‘striking out on one’s own’. Even in cases of disease where accuracy is poorly correlated to expert activity, it is hard to think of cases where these line up orthogonal or worse. Big pharma studies are infamous, but even if you’re in big pharma optimising for ‘can I get evidence to support my product’, your drug actually working does make this easier. Even in pre-replication crisis psychology, true results would be overrepresented versus false ones in the literature compared to some base rate across generated hypotheses.

The ‘residual’ expert class still often remains better. Although most public health experts know little about nicotine per se, there are some nearby health experts, perhaps scattered across our common-sense demarcation of fields, who do know about the impacts of nicotine. It may still take quite a lot of effort to reach parity or superiority to these. Even if we want to strike all theists from free will philosophers, compatibilism does not rise close to unanimity, and so cautions against extremely high confidence this is the correct view.[16] So, I aver, the world is not that mad.

Empirically, the world is mad

One can offer a more direct demonstration of world-madness, and so refute modesty: outperformance.

A common reply is to point to a particular case where those being modest would have gotten it wrong. There are lots of cases where amateurs and mavericks were ridiculed by common sense or experts-at-the-time, only to be subsequently vindicated.

Another problem is the modest view introduces a lag – it seems one often needs to wait for the new information to take root among one’s epistemic peers before changing one’s view, whilst a cogniser just relying on the object level updates on correct arguments ‘at first sight’. It is often crucially important to be fast as well as right in both empirical and moral matters: it is extremely costly if a view makes one slower to recognise (among many other past moral catastrophes) the horror of slavery.

Yet modesty need not infallible, merely an improvement. Citing cases where it goes poorly is (hopefully less than) half the story. Modesty does worse in cases the maverick is right, yet better where the maverick is wrong: there are more cases of the latter than the former. Modesty does worse in being sluggish in responding to moral revolutions, yet better at avoiding being swept away by waves of mistaken sentiment: again, the latter seem more common than the former.[17]

Maybe one can follow a strategy such that you can ‘pick the hits’ of when to carve out exceptions, and so have a superior track record. Yet, empirically, I don’t see it. When I look at people who are touted as particularly good at being ‘correct contrarians’, I see at best something like an ‘epistemic venture capitalist’ – their bold contrarian guesses are right more often than chance, but not right more often than not. They appear by my lights to be unable to judiciously ‘pick their battles’, staking out radical views in topics where there isn’t a good story as to why the experts would be getting this wrong (still less why they’re more likely to get it right). So although they do get big wins, the modal outcome of their contrarian take is a bust.[18] 

Modesty should price in the views of better-than-chance contrarians into how it weighs consensus. Confidence in a consensus view should fall if a good contrarian takes aim at it, but not so much one now takes the contrarian view oneself. If one happens to be a particularly successful contrarian one should follow the same approach: “I get these right surprisingly often, but I’m still wrong more often than not, so it might be worth it to look into this further to see if I can strike gold, but until then I should bank on the consensus view.”

Expert groups are seldom in reflective equilibrium

Even if modesty works well in the ideal case of a clearly identified ‘expert class’, it can get a lot messier in reality:

  1. Suppose one is in the early 1940s and asks, “Is there going to be explosives with many orders of magnitude more power than current explosives?” One can imagine if one consulted explosive experts (however we cash that out), their consensus would generally say ‘no’. If one was able to talk to the physicists working on the Manhattan project, they would say ‘yes’. Which one should an outside view believe?[19]
  2. Most people believe god exists (the so called ‘common consent argument for God’s existence’); if one looks at potential expert classes (e.g. philosophers, people who are more intelligent), most of them are Atheists. Yet if one looks at philosophers of religion (who spend a lot of time on arguments for or against God’s existence), most of them are Theists – but maybe there’s a gradient within them too. Which group, exactly, should be weighed most heavily?

So constructing the ideal ‘weighted consensus’ modesty recommends deferring to can become a pretty involved procedure. One must carefully divine whether a given topic lies closer to the magisterium of one or another putative expert class (e.g. maybe one should lean more to the physicists, as the question is really more ‘about physics’ than ‘about explosives’). One might have to carefully weigh up the relevant epistemic virtues of various expert classes that appear far from reflective equilibrium from one another (so perhaps one might use likely selection effect of philosophy of religion party discount the apparent support this provides). One might have to delve into complicated issues of independence: although most people may believe god exists, unlike guesses of how many skittles are in the jar, they are not all forming this belief independently from one another.[20]

This exercise begins to look increasingly insider-view-esque. Trying to determine the right magisterium involves getting closer to object level considerations about ‘aboutness’ of topics; trying to tease apart issues of independence and selection amount to looking at belief forming practices, and veer close to object level justifications for the belief in question. At some point it becomes extraordinarily challenging to try and back-trace from all these factors to the likely position of the ideal observer: the degrees of freedom these considerations invite (and the challenge in estimating them reliably) make strong modesty go worse.

One should not give up too early, though: modesty can still work pretty well even in these tricky cases. One can ask whether there’s any communication between the classes, and if so any direction of travel (e.g. did some explosive experts end up talking to the physicists, and agreeing they were right? Vice-versa?), even if they were completely isolated, one can ask if a third group having access to both made a decision (e.g. the agreement of the U.S. and German governments with the implied view of the physicists). This is a lot more involved, but the expected ‘accuracy yield per unit time spent’ may still be greater than (for example) making a careful study of the relevant physics. 

A broader modification would be ‘immodest only for the web of belief, but modest for the weights’: one uses an inside view to piece together the graph of considerations around P, but one still defers to consensus on the weights. This may avoid cases where (for example) strong modesty may mistake astronomers as the expert class for about space travel being infeasible (versus primordial rocket scientists), even though astronomers and rocket scientists agreed about the necessary acceleration, but astronomers were inexpert on the key question as to whether that explanation could be produced.[21]

What if one cannot even do that? Then modestly (rightly) offers a counsel of despair. If an area is so fractious there’s no agreement, with no way to see which of numerous of disparate camps have better access the truth of the matter; so suffused with bias that even those with apparent epistemic virtues (e.g. judgement, intelligence, subject-matter knowledge) cannot be seen to even tend towards the truth; what hope does one have to do better than they? In attempting to thread the needle through these hazards towards the right judgement, one will almost certainly run aground somewhere or somehow, alike all one’s epistemic peers or superiors who made the attempt before. Perhaps reality obliges us to undertake these doxastic suicide missions from time to time. If modesty cannot help us, it can at least provide the solace of a pre-emptive funeral, rather than (as immodest views would) cheer us on to our almost certain demise.

Somewhat satisfying Shulman  

Carl Shulman encourages me to offer my credences and rationale in cases he takes to be particularly difficult for my view, and suggests in these cases I either arrive at absurd credences or I am covertly abandoning the strong modesty approach. I offer these below for readers to decide – with the rider that if these are in fact absurd, ‘I’m an idiot’ is a competing explanation to ‘strong modesty is a bad epistemic practice’ (and that, assuredly, whatever one’s credence on the latter, one’s credence in the former should be far greater).

Proposition (roughly)

Credence (ish)

(Modesty-based) rationale, in sketch

Theism

0.1[22]

Mostly discount common consent (non-independence) and PoR (selection). Major hits from more intelligent people/ better informed tend to be atheist, but struggle to extrapolate this closer to 0 given existence proofs of very epistemically virtuous religious people.

Libertarian free will

0.1

Commands a non-trivial minority across virtuous epistemic classes (philosophers, intelligent people, etc), only somewhat degraded by selection worries.

Jesus rose from the dead

0.005

Christianity in particular a very small fraction of possibility space of Theism. Support from its widespread support is mostly (but not wholly) screened off by non-independence effects. Relevant (but distant) expert classes in history etc. weigh adversely.

There has been a case of cold fusion

10^-5

Strong pan scientific consensus against, cold fusion community looks renegade and much less epistemically virtuous. Base rate of these conditional on no effect gives very adverse reference class.

ESP

10^-6

Very strong (but non-complete) trophism among elite common sense, scientists, etc; bad predictive track records for ESP researchers; distant consensuses highly adverse. Some greatly attenuated boost from survey data/small fraction of reasonable believers.

Practical challenges to modesty

Modesty can lead to double-counting, or even groupthink. Suppose in the original example Beatrice does what I suggest and revise their credences to be 0.6, but Adam doesn’t. Now Charlie forms his own view (say 0.4 as well) and does the same procedure as Beatrice, so Charlie now holds a credence of 0.6 as well. The average should be lower: (0.8+0.4+0.4)/3, not (0.8+0.6+0.4)/3, but the results are distorted by using one-and-a-half helpings of Adam’s credence. With larger cases one can imagine people wrongly deferring to hold consensus around a view they should think is implausible, and in general the nigh-intractable challenge from trying to infer cases of double counting from the patterns of ‘all things considered’ evidence.

One can rectify this by distinguishing ‘credence by my lights’ versus ‘credence all things considered’. So one can say “Well, by my lights the credence of P is 0.8, but my actual credence is 0.6, once I account for the views of my epistemic peers etc.” Ironically, one’s personal ‘inside view’ of the evidence is usually the most helpful credence to publicly report (as it helps others modestly aggregate), whilst ones all things considered modest view usually for private consumption.

Community benefits to immodesty

Modesty could be parasitic on a community level. If one is modest, one need never trouble oneself with any ‘object level’ considerations at all, and simply cultivate the appropriate weighting of consensuses to defer to. If everyone free-rode like that, no one would discover any new evidence, have any new ideas, and so collectively stagnate.[23] Progress only happens if people get their hands dirty on the object-level matters of the world, try to build models, and make some guesses – sometimes the experts have gotten it wrong, and one won’t ever find that out by deferring to them based on the fact they usually get it right.[24] 

The distinction between ‘credence by my lights’ versus ‘credence all things considered’ allows the best of both worlds. One can say ‘by my lights, P’s credence is X’ yet at the same time ‘all things considered though, I take P’s credence to be Y’. One can form one’s own model of P, think the experts are wrong about P, and marshall evidence and arguments for why you are right and they are wrong; yet soberly realise that the chances are you are more likely mistaken; yet also think this effort is nonetheless valuable because even if one is most likely heading down a dead-end, the corporate efforts of people like you promises a good chance of someone finding a better path.

Scott Sumner seems to do something similar:

In macro, it’s important for people like me to always search for the truth, and reach conclusions about economic models in a way that is independent of the consensus model. In that way, I play my “worker ant” role of nudging the profession towards a greater truth. But at the same time we need to recognize that there is nothing special about our view. If we are made dictator, we should implement the consensus view of optimal policy, not our own. People have trouble with this, as it implies two levels of belief about what is true. The view from inside our mind, and the view from 20,000 miles out in space, where I see there is no objective reason to favor my view over Krugman’s.

Despite this example, maybe it is the case that ‘having a creative brain which makes big discoveries’ is anticorrelated to ‘having a sober brain well-calibrated to its limitations compared to others’: anecdotally, eccentric views among geniuses are common. Maybe for most it isn’t psychologically tenable to spend one’s life investigating a renegade view one thinks ultimately is likely a dead-end, and in fact people do groundbreaking research generally have to be overconfident to do the best science. If so, we should act communally to moderate this cost, but not celebrate it as a feature.

Not everyone has to do be working on discovering new information. One could imagine a symbiosis between eccentric overconfident geniuses whose epistemic comparative advantage is to who gambol around idea-space to find new considerations, and well-calibrated thoughtful people whose comparative advantage is in soberly weighing considerations to arrive at a well callibrated all-things-considered view.

Conclusion: a pean, and a plea

I have argued above for a strong approach to modesty, one which implies – at least in terms of ‘all things considered view’ – one’s view of the object level merits counts for very little. Even if I am mistaken about the ideal strength of modesty, I am highly confident both the EA and rationalist communities err in the ‘insufficiently modest’ direction. I close on these remarks.

Rationalist/EA exceptionalism 

Both communities endure a steady ostinato of complaints about arrogance. They’ve got a point. I despair of seeing some wannabe-iconoclast spout off about how obviously the solution to some famously recondite issue is X and the supposed experts who disagree obviously just need to better understand the ‘tenets of EA’ or the sequences. I become lachrymose when further discussion demonstrates said iconoclast has a shaky grasp of the basics, that they are recapitulating points already better-discussed in the literature, and so forth.[25]

To stress (and to pre-empt), the problem is not, “You aren’t kowtowing appropriately to social status!” The problem is considerable over-confidence married with inadequate understanding. This both looks bad to outsiders,[26] but it also is bad as the individual (and the community itself) could get to the truth faster if they were more modest about their likely position in the distribution of knowledge about X, and then did commonsensical things to increase it.

Consider Gell-Mann amnesia (via Michael Crichton):

You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the “wet streets cause rain” stories. Paper’s full of them.

In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.

Gell-Mann cases invite inferring adverse judgements based on extrapolating from in instance of poor performance. When experts in multiple different subjects say the same thing (i.e. Murray and Crichton chatted to an expert on Palestine who had the same impression), this adverse inference gets all the stronger.

Some, perhaps many, pieces of work or corporate projects in our community share this property: it might look good or groundbreaking to us as relatively less-informed, domain experts in the fields it touches upon tend to report it is misguided or rudimentary. Although it is possible to indict all these judgements, akin to a person who gives very adverse accounts of all of their previous romantic partners, we may start to wonder about a common factor explanation. Our collective ego is writing checks our epistemic performance (or, in candour, performance generally) cannot cash; general ignorance, rather than particular knowledge, may explain our self-regard.

To discover, not summarise

It is thought that to make the world go better new things need to be discovered, above and beyond making sound judgements on existing knowledge. Quickly making accurate determinations of the balance of reason for a given issue is greatly valuable for the latter, but not so much for the former.

Yet the two should not be confused. If one writes a short overview of a subject ‘for internal consumption’ which gives a fairly good impression of what a particular view should be, one should not be too worried if a specialist complains that you haven’t covered all the topics as adequately as one might. However, if one is aiming to write something which articulates an insight or understanding not just novel to the community, but novel to the world, one should be extremely concerned if domain experts review this work and say things along the lines of, “Well, this is sort of a potted recapitulation of work in our field, and this insight is widely discussed”.

Yet I see this happen a lot to things we tout as ‘breakthrough discoveries’. We want to avoid case where we waste our time in unwitting recapitulation, or fail to catch elementary mistakes. Yet too often we license ourselves to pronounce these discoveries without sufficient modesty in cases where there’s already a large expert community working on similar matters. This does not preclude these discoveries, but it cautions us to carefully check first. On occasions where I take myself to have a new insight in areas outside my field (most often philosophy), I am extremely suspect of my supposed discovery: all too often would this arise from my misunderstanding, or already be in the literature somewhere I haven’t looked. I carefully consult the literature as best as I can, and run the idea by true domain experts, to rule out these possibilities.[27] 

Others seem to lack this modesty, and so predictably err. More generally, a more modest view of ‘intra-community versus outside competence’ may also avoid cases of having to reinvent the wheel (e.g. that scoring rule you spent six months deriving for a karma system is in this canonical paper), or for an effort to derail (e.g. oh drat, our evaluation provides worthless data because of reasons we could have known from googling ‘study design’).

Paradoxically pathological modesty 

If the EA and rationalist communities comprised a bunch of highly overconfident and eccentric people buzzing around bumping their pet theories together, I may worry about overall judgement and how much novel work gets done, but I would at grant this at least looks like fertile ground for new ideas to be developed.

Alas, not so much. What occurs instead is agreement approaching fawning obeisance to a small set of people the community anoints as ‘thought leaders’, and so centralizing on one particular eccentric and overconfident view.[28] So although we may preach immodesty on behalf of the wider community, our practice within it is much more deferential.

I hope a better understanding of modesty can get us out of this ‘worst of both worlds’ scenario. It can at least provide better ‘gurus’ to defer to. Better, modesty also helps to correct two mistaken impressions: one, overly wide gap between our gurus and other experts; two, the overly narrow gap between ‘intelligent layperson in the community’ and ‘someone able to contribute to the state of the art’. Some topics are really hard: being able to become someone with ‘something useful to say’ about these not take days but take years; there are many deep problems we must concern ourselves with; that the few we select as champions, despite their virtue, cannot do them all alone; and that we need all the outside help we can get.

Coda

What the EA community mainly has now is a briar-patch of dilettantes: each ranges widely, but with shallow roots, forming whorls around others where it deems it can find support. What it needs is a forest of experts: each spreading not so widely; forming a deeper foundation and gathering more resources from the common ground; standing apart yet taller, and in concert producing a verdant canopy.[29] I hope this transformation occurs, and aver modesty may help effect it.

  

Acknowledgements

I thank Joseph Carlsmith, Owen Cotton-Barratt, Eric Drexler, Ben Garfinkel, Roxanne Heston, Will MacAskill, Ben Pace, Stefan Schubert, Carl Shulman, and Pablo Stafforini for their helpful discussion, remarks, and criticism. Their kind help does not imply their agreement. The errors remain my own.

[Edit 30/10: Rewording and other corrections – thanks to Claire Zabel and Robert Wiblin]

  


[1] Much of this follows discussion in the social epistemology literature about conciliationism, or the ‘equal weight view’. See here for a summary

[2] They also argue at length about the appropriate weight each of these considerations should have on the scales of judgement. I suggest (although this is not necessary for this argument) that in many cases most of the action lies in judging the ‘power’ of evidence. In most cases I observe people agree that a given consideration C influences the credence one holds in P; they usually also agree in its qualitative direction; the challenge comes in trying to weigh each consideration against the others, to see which considerations one’s credence over P should pay the greatest attention to.

This may represent a general feature of webs of belief being dense and many-many (A given credence is influenced by many other considerations, and forms a consideration for many credences in turn), or it may simply be a particular feature of webs of belief in which humans perform poorly: although I am confident I can determine the sign of a particular consideration, I generally don’t back myself to hold credences (or likelihood ratios) to much greater precision than the first significant digit, and I (and, perhaps, others) struggle in cases where large numbers of considerations point in both directions.  

[3] In the literature this is called ‘straight averaging’. For a variety of technical reasons this doesn’t quite work as a peer update rule. That said, given things like bayesian aggregation remain somewhat open problems, I hope readers will accept my promissory note that there will be a more precise account which will produce effectively the same results (maybe ‘approximately splitting the difference’) through the same motivation.

[4] C.f. Aumann’s agreement theorem. As an aside (which I owe to Carl Shulman), straight averaging will not work in some degenerate cases where (similar to ‘common knowledge puzzles’) one can infer precise observations from the probabilities stated. The neatest example I can find comes from Hal Finney (see also):

Suppose two coins are flipped out of sight, and you and another person are trying to estimate the probability that both are heads. You are told what the first coin is, and the other person is told what the second coin is. You both report your observations to each other.

Let’s suppose that they did in fact fall both heads. You are told that the first coin is heads, and you report the probability of both heads as 1/2. The other person is told that the second coin is heads, and he also reports the probability as 1/2. However, you can now both conclude that the probability is 1, because if either of you had been told that the coin was tails, he would have reported a probability of zero. So in this case, both of you update your information away from the estimate provided by the other.

[5] To motivate: Adam and Beatrice no longer know whether or not reasons they hold for or against P are private evidence or not. Yet (given epistemic peerhood), they have no principled reason to suppose “I know something that they don’t” is more plausible than the opposite. So again they should be symmetrical.

[6] (On which more later) it is worth making clear that the possibility of bias for either Adam or Beatrice doesn’t change the winning strategy on expectation. Say Adam’s credence for P is in fact biased upwards by 0.4. If Adam knows this, he can adjust and become unbiased, if Oliver or Beatrice knows this (and knows Adam doesn’t), the break the peerhood for Adam but can simulate unbiased Adam* which would remain a peer, and act accordingly. If none of them know this, then it is the case that Beatrice wins, as does Oliver following a non-averaging ‘go with Beatrice’ strategy. Yet this is simply epistemic luck: without information, all reasonable prior distribution candidates of (Adam’s bias – Beatrice’s bias) are symmetrical about 0.  

[7] Another benefit of modesty is speed: Although it is the case Adam and Beatrice’s credence (and thus the average) gets more accurate if they have time to discuss it, and so catch one another if they make a mistake or reveal previously-private evidence, averaging is faster and the trade-off in time for better precision may not be worth it. It still remains the case, as per the first example, that they still do better, after this discussion, if they meet in the middle on residual disagreement. 

[8] A further (albeit minor and technical) dividend is that although individual guesses may form any distribution (for which the standard deviation may not be a helpful summary), the central limit theorem applies to the average of guesses distribution, so it tends to normality.  

[9] Even if one is the world authority, there should be some deference to lesser experts. In cases where the world expert is an outlier, one needs to weigh up numbers versus (relative) epistemic superiority to find the appropriate middle.

[10] 

God from the Mount of Sinai, whose gray top
Shall tremble, he descending, will himself
In Thunder Lightning and loud Trumpets sound
Ordaine them Lawes…

Milton, Paradise Lost

[11] I take the general pattern that strong modesty usually immures one from common biases is a further point in its favour.  

[12] I owe this to Eric Drexler

[13] A related philosophical defence would point out that the self-undermining objection would only apply to whether one should believe modesty, not whether modesty is in fact true.

[14] I naturally get much more sceptical if that person then generalises from this N=1 uncontrolled unblinded crossover trial to others, or takes it as lending significant support against some particular expert consensus or expertise more broadly: “Doctors don’t know anything about back pain! They did all this rubbish but I found out all anyone needs to do is cut carbs!”

[15] It also provokes fear and trembling in my pre-paradigmatic day job, given I don’t want the area to have strong founder effects which poorly track the truth.

[16] For example:

One of the easiest hard questions, as millennia-old philosophical dilemmas go. Though this impossible question is fully and completely dissolved on Less Wrong, aspiring reductionists should try to solve it on their own.

[17] Aside: A related consideration is ‘optimal damping’ of credences, which is closely related to resilience. Very volatile credences may represent the buffeting of a degree of belief by evidence large relative to one’s prior – but it may also represent poor calibration in overweighing new evidence (and vice versa). The ‘ideal’ response in terms of accuracy is given by standard theory. Yet it is also worth noting that’s one prudential reasons may want to introduce further lag or lead, akin to the ‘D’ or ‘I’ components of a PID controller. In large irreversible decisions (e.g. career choice) it may be better to wait a while after one’s credences support a change to change action; for case of new moral consideration it may be better to act ‘in advance’ for precautionary principle-esque reasons.

[18] (Owed to Will MacAskill) There’s also a selection effect: of a sample of ‘accurate contrarians’, many of these may be lucky rather than good.

[19] I owe this particular example to Eric Drexler, but similar counter-examples along these lines to Carl Shulman.

[20] Another general worry is these difficult-to-divine considerations offer plenty of fudge factors – both to make modesty get the ‘right answer’ in historical cases, and to fudge present areas of uncertainty to get results that accord with one’s prior judgement.

[21] I owe both this modification and example to discussions with Eric Drexler. There are some costs – one may think there are cases one should defer to an outside view on the web of belief (E.g. Christian apologist: “Sure, I agree with scientific consensus that it’s improbable Jesus rose naturally from the dead, but the key argument is whether Jesus rose supernaturally from the dead. So the consensus for philosophers of religion is the right expert class.”) The balance of merit overall is hard to say, but such a modification still looks like pretty strong modesty.

[22] In conversation I recall a suggestion by Shulman such a credence should change one’s behaviour regarding EA – maybe one should do theology research in the hope of finding a way to extract infinite value etc. Yet the expert class for action|Theism gives a highly adverse prior: virtually no actual theists (regardless of theological expertise, within or outside EA) advocate this.

[23] I understand a similar point is raised in economics regarding the EMH and the success of index funds. Someone has to do the price discovery.

[24] I owe this mainly to Ben Pace, Andrew Critch argues similarly.

[25] For obvious reasons I’m reluctant to cite specific examples. I can offer some key words for the sort of topics I see this problem as endemic: Many-worlds, population ethics, free will, p-zombies, macroeconomics, meta-ethics.

[26] C.f. Augustine, On the Literal Meaning of Genesis:

Usually, even a non-Christian knows something about the earth, the heavens, and the other elements of this world, about the motion and orbit of the stars and even their size and relative positions, about the predictable eclipses of the sun and moon, the cycles of the years and the seasons, about the kinds of animals, shrubs, stones, and so forth, and this knowledge he hold to as being certain from reason and experience. Now, it is a disgraceful and dangerous thing for an infidel to hear a Christian, presumably giving the meaning of Holy Scripture, talking nonsense on these topics; and we should take all means to prevent such an embarrassing situation, in which people show up vast ignorance in a Christian and laugh it to scorn.

[27] I’m uncommonly fortunate that for me such domain experts are both nearby and generous with their attention. Yet this obstacle is not insurmountable. An idea (which I owe to Pablo Stafforini) is that a contrarian and a sceptic of the contrarian view could bet on whether a given expert, on exposure to the contrarian view, would change their mind as the contrarian predicts. S may bet with C: “We’ll pay some expert $X to read your work explicating your view, if they change their mind significantly in favour (however we cash this out) I’ll pay the $X, if not, you pay the $X.

[28] C.f. Askell’s and Page’s remarks on ‘buzz’.

[29] Perhaps unsurprisingly, I would use a more modest ecological metaphor in my own case. In reclaiming extremely inhospitable environments, the initial pioneer organisms die rapidly. Yet their corpses sustain detritivores, and little by little, an initial ecosystem emerges to be succeeded by others. In a similar way, I hope that the detritus I provide will, after a fashion (and a while), become the compost in which an oak tree grows.  

 

In defence of democracy

Last week the UK held a referendum on whether it should remain a member of the European Union. ‘Leave’ won by a narrow (52%) majority. The aftermath so far involves the resignation of the Prime Minister and consequent leadership election; a leadership challenge within the opposition; an increasingly restive Scotland looking for independence; and large slides of UK stocks and currency.

Contra the balance of the chattering classes (and the great majority of all those in my social media bubble). I think the results of the referendum should be respected, and therefore the UK should leave the EU.[1]

“Democratic” “legitimacy”

The people of England regards itself as free; but it is grossly mistaken; it is free only during the election of members of parliament. As soon as they are elected, slavery overtakes it, and it is nothing.

Rousseau – The Social Contract

By and large, one’s attitude to direct democracy waxes and wanes with whether the people vote for the things you want. Supportive referenda are imbued with the finality that the people – surely the highest authority in a democracy! – have spoken. In contrast, regrettable referenda are not the last word: as has been pointed out ad nauseum, referenda have no constitutional force in the UK, and are only advisory. Thus remainers dare to hope that Brexit can yet be defeated in parliament (a large majority of MPs preferred to remain).

However, the current government in our representative democracy was elected on a manifesto that prominently included offering an ‘in-out’ referendum on EU membership, and that it would abide by the result:

We will legislate in the first session of the next Parliament for an in-out referendum to be held on Britain’s membership of the EU before the end of 2017. We will negotiate a new settlement for Britain in the EU. And then we will ask the British people whether they want to stay in on this basis, or leave. We will honour the result of the referendum, whatever the outcome.

Conservative party manifesto (my emphasis)

Given the difficult-to-reverse nature of a ‘Leave’ option, perhaps there should have been further barriers to Leave than a bare majority: a super-majority, repeated referenda several years apart, or a requirement it passes in each constituent nation of the UK, inter alia. Yet to the ardent supporter of representative democracy, this is simply a matter between the parties and the electorate: the former make their proposals on this point; the latter can weigh the relative merits in deciding who to vote for. I imagine further qualifications along these lines would not have been vote-winners.

The elected government offered an advisory referendum (bare majority warts and all) as a decision rule for subsequent parliamentary action. Claiming this is democratically illegitimate entails not only opposition to direct democracy, but opposition to representatives using referenda to guide their decisions even when this would be popular to the electorate.[2] Most of us regard parties who betray manifesto commitments dimly, approximately commensurate to how prominently these commitments were made. Of course, other parties have no obligation to abide by the Conservative manifesto, but the Conservative government and MPs should. For the Conservative government to not subsequently table a bill to leave the EU is a manifesto betrayal considerably worse than the Liberal Democrats on tuition fees.[3] The same applies to a conservative MP not voting with the government – and they have an absolute majority sufficient to drive Brexit[4] (and opposition to leaving is not unanimously held by representatives from other parties).

Contra plebiscites (and contra plebs?)

Suppose a man was in charge of a large and powerful animal, and made a study of its moods and wants; he would learn when to approach and handle it, when and why it was especially savage or gentle, what the different noises it made meant, and what tone of voice to use to soothe or annoy it… But he would not really know which of the creature’s tastes and desires was admirable or shameful, good or bad, right or wrong; he would simply use the terms on the basis of its reactions, calling what pleased it good, what annoyed it bad.

Plato – The Republic

Referenda can be legitimate but nonetheless irresponsible. We may assert many issues are ‘too complicated‘ to be adjudicated by the population. The prevailing norm of whatever irrationalities, prejudices and misinformation exists in the population is not a ‘wisdom of crowds’, and an individual voter cannot be trusted to vote even in accordance with their own interests – leave alone the best interests of all. A referendum just offers people the opportunity to punch themselves in the face.

The technocrats (of which I am in the larval stages of becoming) will do much better. I work in public health, and I dread to see what would result if we made all of our decisions by popular vote. Far better to limit political interference: perhaps ask the population what values they want to see in the health service, and leave the implementation up to us (and perhaps not even that). So too, I assume, other areas of government.

But so too whether to be a member of the European Union? Granted the consequences are dazzlingly complicated: beyond internationalist impulses, I voted remain mainly in fealty to the balance of expert opinion (and – like the average voter – I deem myself better informed than the average voter). Yet issues of wide-reaching importance perhaps should be the subject of referendum in addition to representative democracy: few people find a party that exactly mirrors their view on all issues, and so chosing who to vote into government is usually a compromise. Having a referendum on the key issue avoids the circumstance where the majority are opposed, but voted for the governing party due to agreement on other issues, and thus getting a direct mandate on the question of issue is superior.[5]

Whether to become an independent nation is similarly complicated – yet it is anathema to self-determination to claim that the population do not have the final say on whether to be independent (c.f. Scotland). If anything, which party to vote for in the general election is more complicated than the choice presented in the EU referendum: it demands assessing the merits of each party’s view on the EU, as well as on many other similarly complicated issues, and then aggregating across them to select one’s most preferred party overall. If it is irresponsible to ask them to choose in a referendum, is it not similarly irresponsible to ask them to elect representatives?

In any case, I think the average man on the street was competent to vote on the ‘EU question’, and they probably voted rationally in accordance with their convictions.[6]

Will of the people [sic]

After the uprising of the 17th of June

The Secretary of the Writers’ Union

Had leaflets distributed in the Stalinallee

Stating that the people

Had forfeited the confidence of the government

And could win it back only

By redoubled efforts. Would it not be easier

In that case for the government

To dissolve the people

And elect another?

Brecht – The Solution

Favourable votes engender paeans to the good sense of the common man. Unfavourable votes – alas – engender an equal and opposite impulse. Both are similarly inaccurate; the latter especially toxic.

It didn’t take long after the results to start castigating sections of the electorate that didn’t vote the right way as democratically incompetent. The ‘grey vote’ tended to vote leave,[7] so we regret enfranchising these people, soon to die, because they availed themselves of the opportunity to spitefully ruin our futures in a pre-mortem Parthian shot (a pity we did not recognize this great democratic injustice at the Scottish Independence referendum two years ago, where the same issue delivered our preferred result). The poorer, less well-educated, and those in lower socio-economic strata also tended to vote leave – cue remarks (often from those previously worried about social justice and iniquities in educational opportunity) about how these trends are proof positive about those voting leave were sufficiently stupid their say should be disregarded.

Among the more sophisticated, the urge to castigate transmogrifies into the urge to reinterpret, to arrogate, and to patronize. The working classes, the lament goes, are left behind by the economy and stagnant or worsening material conditions. Thus the poor impressionable masses turn their ears to nasty ‘populists’ who scapegoat immigrants and the EU for all their woes. Although the vox populi said to leave the European Union, what was really meant was a howl of despair at their economic circumstances. Solve this, and they shall develop informed and enlightened attitudes (i.e. our own).

Yet Remain slightly won on the economy – in the whole population slightly more people thought it would be bad than would be good (albeit they divided much more evenly than economists, who came out strongly on the ‘bad’ side). Via Ashcroft, we find the most important reason to leavers was national sovereignty, with immigration second (other polls put the order the other way around). Leave and Remain voters did divide right versus left, but on social issues: among those who thought multiculturalism was a force for ill, a considerable majority voted leave; out of those who thought it was a force for good, a considerable majority voted remain, mutatis mutandis for other items in the social liberal itinerary (feminism, the green movement) and social liberalism itself. In contrast, the pro- and anti- capitalists divided evenly between Leave and Remain.

Getting from this to the, ‘it was really because they were fed up with being poor’ looks a stretch.[8] By contrast, taking the people at their word is looks more parsimonious. The narrow majority who voted to leave want to keep a nation with absolute control over its own affairs, rather than surrendering some part of this to political apparatus that is not entirely British. They broadly dislike the ‘progress’ of progressivism. They may in part dislike immigration due to economic impacts, but perhaps in greater part they dislike it because they see immigrants as diluting or undermining a British national character they want to preserve.

Given these objectives, voting to leave the EU seems an instrumentally rational means to achieve them. In the same way remainers are confident a vote to leave is a step backwards for the country, they can be confident it is a step in the right direction. Far from being an epiphenomenon of economic woe, it looks plausible that for many voting leave any economic costs would be a price worth paying to move toward these social objectives.

Wherefore democracy?

I’m (broadly) consequentialist, so I don’t see any fundamental value in democracy, or some free-standing value in the oft-reified ‘will of the people’. Yet its instrumental value looks considerable: democratic societies tend not to war, tend to be prosperous, and have a good track record of securing the wellbeing of their citizens across most measures. Perhaps this correlation is confounded – democracy emerges as a result of prosperity and social progress, and not the other way around. The story for a direct role is pretty good, though: if the people get to decide on the government, the latter has an interest to act in the interests of the former, and not do something transparently harmful.

Many things in the world are not transparent, and letting the ‘people decide’ can be tantamount to letting the lunatics run the asylum. Perhaps we should dial back people power, and thus representative democracy allows us to have government by second order technocracy: let them get on with deciding things in our interests, and the power of election held over them is a check on them doing anything egregiously bad.

Perhaps. Yet this ethos is particularly challenging when the voting is on values rather than facts about the world. The referendum reveals a deeply divided society, and may yet be the touchstone to a political realignment. Many have found the discovery of a large swell of anti-immigration, anti-internationalist and social conservative sentiment very unwelcome (I sympathize – I now call myself British only with a stutter). One wonders whether the newfound trust now bestowed on the political representatives to decide these matters over and above the popular will represents a belated realization that the political class was always on our side, and could be trusted to broadly follow our agenda – no matter who they were ostensibly representing. (One could also wonder exactly how badly these previously disenfranchised groups will react when even a referendum cannot get them what they want).

I hope that the great majority can be persuaded of the benefits of supranational governance and broad international cooperation, that immigration is usually a matter of mutual benefit, and that Britishness can rise higher than petty paroxysms of parochial prejudice. Until then, I have to find some way of getting along with those who think the opposite. There are alternatives to settling disagreements at the ballot box. They are almost always worse.

It is easy to be a democrat when you know the people are on your side; the arc of history bends towards you; and you see your opponents aims crushed to the lamentations of their punditry. This exhilaration of political power does not distinguish democracy. What does is how you respond when the people are ranged against you; when you see society veer off course; and when they strike against the things you hold dear.

One’s respect for democratic principles (referenda included) should arise in consonance with the understanding that they tend to effect peace, prosperity, and happiness; and are best at producing harmonious government and civil society from often insoluble popular discord. To modulate one’s respect for these principles by the results of a particular instance damages this important political timbre: why should they acquiesce to the majority, if they know you would not do the same?

For all its goods, democracy is not sacrosanct. There are some even more important things for which, if necessary, it should be sacrificed. Membership of the European Union is not one of them.


[1] For whatever it is worth, although it should be irrelevant: I voted remain. I campaigned (albeit briefly) on behalf of remain. I think leaving the EU is disastrous. I would guess it is the worst decision the country has made since the Iraq war, and may be judged by posterity to be the worst in the entire post-war period up until now. I hope not.

[2] I am unusually confident despite my charitable reflexes arguments here are gratuitously self-serving. Consider this reversal: the Conservatives ran on a manifesto to removing the UK from the EU without any referendum, and won a majority. Be honest: would one really say that was more democratic, and not be on the streets demanding a referendum (which, if a bare majority voted to remain, one would expect the government to honour and not pass a bill to leave the EU anyway)?

[3] The lib dems could offer in partial exculpation that they were in a coalition government as a junior partner, and could not be expected to honour all their manifesto commitments (to which the reply is that the tuition fees pledge was such a key part of their manifesto – signed pledges and all – that it should have either been one of the last things to negotiate away, or that it should be a ‘red line’ to forming a coalition in the first place). No such excuses present themselves for the Conservatives.

[4] One reply could be that the use of First Past the Post makes this majority illegitimate – the Conservatives only won 30-ish percent of the popular vote. Perhaps. But the main loser from FPTP was UKIP whose 12.6% of vote yielded 1 seat (on Proportional Representation it would be 82), and UKIP were also keen on an ‘in-out’ referendum. The combined vote share of UKIP and the Conservatives was 49.5%, Labour, Lib Dems, SNP and Greens amassed 46.8%. What government would have emerged from this divided popular vote is unclear, and likewise whether an EU referendum would have been one of the horses traded into or out of the plausible coalitions or confidence and supply arrangements (the challenge of holding political parties accountable for what they end up enacting in coalitions and difficulty of decisively voting a party into or out of power are often deemed disadvantages of PR), but it seems fairly likely an ‘in-out’ referendum would have remained on the agenda.

Naturally, a criticism along these lines would only show that in this particular case a referendum was ‘undemocratic’, and such a referendum could be legitimate if it emerged via one’s preferred system of allocating representatives to votes.

[5] Notably, even UKIP, fairly close to a ‘single issue party’ (leave the EU!) nonetheless proposed a referendum on EU membership in their manifesto.

[6] A lot has been made over misinformation by the leave campaign. Particularly the ‘350m’ to the EU which could be sent to the NHS, or that it may not reduce immigration in absolute terms. I am not sure these count for much. The UK is a net contributor to the EU (although Remain would say it is a good deal on net), and whether it is 350m versus 200m per week seems unlikely to be material to voters. Likewise having greater control of immigration via release of free-movement obligations within the EU seems preferable for those opposed to immigration, even it doesn’t immediately effect a reduction – one might simply want to be more selective about who ‘gets in’. The accusations about misinformation can go both ways – the chancellor’s projections of the economic damage are significantly greater than those of most of economic bodies, for example.

Granted, reasonable voters likely should be weighing multiple factors in their decision, so if the economic costs are undersold whilst the changes to immigration are oversold, one could be misinformed (as an extreme example, I doubt many but the most ardent opponents of immigration would vote to leave if it dropped GDP by half). As yet, evidence for result-differing ‘Breget’ is scant. A re-poll projects 1.2m reject voting leave – but 0.4m regret voting remain.

[7] Although the ‘grey vote’ also tended to vote simpliciter: the absolute proportion of elderly voting to remain out of eligible voters was greater than younger cohorts.

[8] Granted, leavers tended to be more pessimistic than remainers (but not dramatically so) about their economic prospects. Yet the main split by pessimism was in general rather than economic terms (e.g. ‘Overall, life in britain is worse than it was 30 years ago’). This is hardly incongruent with the other results: the social issues leavers tend to identify as forces for ill have been in the ascendancy over this period. I’d be pretty down on the prospects for my country if the hard right were in government for the foreseeable future, even if the economic forecast was incandescent.

Are History’s “Greatest Philosophers” All That Great?

Introduction

In the canon of western philosophy, generally those regarded as the ‘greatest’ philosophers tend to live far in the past. Consider this example from an informal poll:

  1. Plato (428-348 BCE)
  2. Aristotle (384-322 BCE)
  3. Kant (1724-1804)
  4. Hume (1711-1776)
  5. Descartes (1596-1650)
  6. Socrates (469-399 BCE)
  7. Wittgenstein (1889-1951)
  8. Locke (1632-1704)
  9. Frege (1848-1925)
  10. Aquinas (1225-1274)

(source: LeiterReports)

I take this as fairly representative of consensus opinion—one might argue about some figures versus those left out, or the precise ordering, but most would think (e.g.) Plato and Aristotle should be there, and near the top. All are dead, and only two were alive during the 20th century.

But now consider this graph of human population over time (US Census Bureau, via Wikipedia):

1000px-Population_curve.svg

The world population at 500BCE  is estimated to have been 100 million; in the year 2000, it was 6.1 billion, over sixty times greater. Thus if we randomly selected people from those born since the ‘start’ of western philosophy, they would generally be born close to the present day. Yet when it comes to ‘greatest philosophers’, they were generally born much further in the past than one would expect by chance. Continue reading “Are History’s “Greatest Philosophers” All That Great?”

Beware surprising and suspicious convergence

Imagine this:

Oliver: … Thus we see that donating to the opera is the best way of promoting the arts.

Eleanor: Okay, but I’m principally interested in improving human welfare.

Oliver: Oh! Well I think it is also the case that donating to the opera is best for improving human welfare too.

Generally, what is best for one thing is usually not the best for something else, and thus Oliver’s claim that donations to opera are best for the arts and human welfare is surprising. We may suspect bias: that Oliver’s claim that the Opera is best for the human welfare is primarily motivated by his enthusiasm for opera and desire to find reasons in favour, rather than a cooler, more objective search for what is really best for human welfare.

The rest of this essay tries to better establish what is going on (and going wrong) in cases like this. It is in three parts: the first looks at the ‘statistics’ of convergence – in what circumstances is it surprising to find one object judged best by the lights of two different considerations? The second looks more carefully at the claim of bias: how it might be substantiated, and how it should be taken into consideration. The third returns to the example given above, and discusses the prevalence of this sort of error ‘within’ EA, and what can be done to avoid it.

Varieties of convergence

Imagine two considerations, X and Y, and a field of objects to be considered. For each object, we can score it by how well it performs by the lights of the considerations of X and Y. We can then plot each object on a scatterplot, with each axis assigned to a particular consideration. How could this look?

Convergence

At one extreme, the two considerations are unrelated, and thus the scatterplot shows no association. Knowing how well an object fares by the lights of one consideration tells you nothing about how it fares by the lights of another, and the chance that the object that scores highest on consideration X also scores highest on consideration Y is very low. Call this no convergence.

At the other extreme, considerations are perfectly correlated, and the ‘scatter’ plot has no scatter, but rather a straight line. Knowing how well an object fares by consideration X tells you exactly how well it fares by consideration Y, and the object that scores highest on consideration X is certain to be scored highest on consideration Y. Call this strong convergence.

In most cases, the relationship between two considerations will lie between these extremes: call this weak convergence. One example is there being a general sense of physical fitness, thus how fast one can run and how far one can throw are somewhat correlated. Another would be intelligence: different mental abilities (pitch discrimination, working memory, vocabulary, etc. etc.) all correlate somewhat with one another.

More relevant to effective altruism, there also appears to be weak convergence between different moral theories and different cause areas. What is judged highly by (say) Kantianism tends to be judged highly by Utilitarianism: although there are well-discussed exceptions to this rule, both generally agree that (among many examples) assault, stealing, and lying are bad, whilst kindness, charity, and integrity are good.1 In similarly broad strokes what is good for (say) global poverty is generally good for the far future, and the same applies for between any two ‘EA’ cause areas.2

In cases of weak convergence, points will form some some sort of elliptical scatter, and knowing how an object scores on X does tell you something about how well it scores on Y. If you know that something scores highest for X, your expectation of how it scores for Y should go upwards, and the chance of it also scores highest for Y should increase. However, the absolute likelihood of it being best for X and best for Y remains low, for two main reasons:

divergence

Trade-offs: Although consideration X and Y are generally positively correlated, there might be a negative correlation at the far tail, due to attempts to optimize for X or Y  at disproportionate expense for Y or X. Although in the general population running and throwing will be positively correlated with one another, elite athletes may optimize their training for one or the other, and thus those who specialize in throwing and those who specialize in running diverge. In a similar way, we may think believe there is scope for similar optimization when it comes to charities or cause selection.

regressexp

Chance: (c.f.) Even in cases where there are no trade-offs, as long as the two considerations are somewhat independent, random fluctuations will usually ensure the best by consideration X will not be best by consideration Y. That X and Y only weakly converge implies other factors matter for Y besides X. For the single object that is best for X, there will be many more not best for X (but still very good), and out of this large number of objects it is likely one will do very well on these other factors to end up the best for Y overall. Inspection of most pairs of correlated variables confirms this: Those with higher IQ scores tend to be wealthier, but the very smartest aren’t the very wealthiest (and vice versa), serving fast is good for tennis, but the very fastest servers are not the best players (and vice versa), and so on. Graphically speaking, most scatter plots bulge in an ellipse rather than sharpen to a point.

The following features make a single object scoring highest on two considerations more likely:

  1. The smaller the population of objects. Were the only two options available to OIiver and Eleanor, “Give to the Opera” and “Punch people in the face”, it is unsurprising the former comes top for many considerations.
  2. The strength of their convergence. The closer the correlation moves to collinearity, the less surprising finding out something is best for both. It is less surprising the best at running 100m is best at running 200m, but much more surprising if it transpired they threw discus best too.
  3. The ‘wideness’ of the distribution. The heavier the tails, the more likely a distribution is to be stretched out and ‘sharpen’ to a point, and the less likely bulges either side of the regression line are to be populated. (I owe this to Owen Cotton-Barratt)

In the majority of cases (including those relevant to EA), there is a large population of objects, weak convergence and (pace the often heavy-tailed distributions implicated) it is uncommon for one thing to be best b the lights of two weakly converging considerations.

Proxy measures and prediction

In the case that we have nothing to go on to judge what is good for Y save knowing what is good for X. Our best guess for what is best for Y is what is best for X. Thus the Opera is the best estimate for what is good for human welfare, given only the information that it is best for the arts. In this case, we should expect our best guess to be very likely wrong. Although it is more likely than any similarly narrow alternative (“donations to the opera, or donations to X-factor?”) Its absolute likelihood relative to the rest of the hypothesis space is very low (“donations to the opera, or something else?”).

Of course, we usually have more information available. Why not search directly for what is good for human welfare, instead of looking at what is good for the arts? Often searching for Y directly rather than a weakly converging proxy indicator will do better: if one wants to select a relay team, selecting based on running speed rather than throwing distance looks a better strategy. Thus finding out a particular intervention (say the Against Malaria Foundation) comes top when looking for what is good for human welfare provides much stronger evidence it is best for human welfare than finding out the opera comes top when looking for what is good for a weakly converging consideration. 3

Pragmatic defeat and Poor Propagation

Eleanor may suspect bias is driving Oliver’s claim on behalf of the opera. The likelihood of the opera being best for both the arts and human welfare is low, even taking their weak convergence into account. The likelihood of bias and motivated cognition colouring Oliver’s judgement is higher, especially if Oliver has antecedent commitments to the opera. Three questions: 1) Does this affect how she should regard Oliver’s arguments? 2) Should she keep talking to Oliver, and, if she does, should she suggest to him he is biased? 3) Is there anything she can do to help ensure she doesn’t make a similar mistake?

Grant Eleanor is right that Oliver is biased. So what? It entails neither he is wrong nor the arguments he offers in support are unsound: he could be biased and right. It would be a case of the genetic fallacy (or perhaps ad hominem) to argue otherwise. Yet this isn’t the whole story: informal ‘fallacies’ are commonly valuable epistemic tools; we should not only attend to the content of arguments offered, but argumentative ‘meta-data’ such as qualities of the arguer as well.4

Consider this example. Suppose you are uncertain whether God exists. A friendly local Christian apologist offers the reasons why (in her view) the balance of reason clearly favours Theism over Atheism. You would be unwise to judge the arguments purely ‘on the merits’: for a variety of reasons, the Christian apologist is likely to have slanted the evidence she presents to favour Theism; the impression she will give of where the balance of reason lies will poorly track where the balance of reason actually lies. Even if you find her arguments persuasive, you should at least partly discount this by what you know of the speaker.

In some cases it may be reasonable to dismiss sources ‘out of hand’ due to their bias without engaging on the merits: we may expect the probative value of the reasons they offer, when greatly attenuated by the anticipated bias, to not be worth the risks of systematic error if we mistake the degree of bias (which is, of course, very hard to calculate); alternatively, it might just be a better triage of our limited epistemic resources to ignore partisans and try and find impartial sources to provide us a better view of the balance of reason.

So: should Eleanor stop talking to Oliver about this topic? Often, no. First (or maybe zeroth), there is the chance she is mistaken about Oliver being biased, and further discussion would allow her to find this out. Second, there may be tactical reasons: she may want to persuade third parties to their conversation. Third, she may guess further discussion is the best chance of persuading Oliver, despite the bias he labours under. Fourth, it may still benefit Eleanor: although bias may undermine the strength of reasons Oliver offers, they may still provide her with valuable information. Being too eager to wholly discount what people say based on assessments of bias (which are usually partly informed by object level determinations of various issues) risks entrenching one’s own beliefs.

Another related question is whether it is wise for Eleanor to accuse Oliver of bias. There are some difficulties. Things that may bias are plentiful, thus counter-accusations are easy to make: (“I think you’re biased in favour of the opera due to your prior involvement”/”Well, I think you’re biased against the opera due to your reductionistic and insufficiently holistic conception of the good.”) They are apt to devolve into the personally unpleasant (“You only care about climate change because you are sleeping with an ecologist”) or the passive-aggressive (“I’m getting really concerned that people who disagree with me are offering really bad arguments as a smokescreen for their obvious prejudices”). They can also prove difficult to make headway on. Oliver may assert his commitment was after his good-faith determination that opera really was best for human welfare and the arts. Many, perhaps most, claims like these are mistaken, but it can be hard to tell (or prove) which.5

Eleanor may want to keep an ‘internal look out’ to prevent her making a similar mistake to Oliver. One clue is a surprising lack of belief propagation: we change our mind about certain matters, and yet our beliefs about closely related matters remain surprisingly unaltered. In most cases where someone becomes newly convinced of (for example) effective altruism, we predict this should propagate forward and effect profound changes to their judgements on where to best give money or what is the best career for them to pursue. If Eleanor finds in her case that this does not happen, that in her case her becoming newly persuaded by the importance of the far future does not propagate forward to change her career or giving, manifesting instead in a proliferation of ancillary reasons that support her prior behaviour, she should be suspicious of this surprising convergence between what she thought was best then, and what is best now under considerably different lights.

EA examples

Few Effective altruists seriously defend the opera as a leading EA cause. Yet the general problem of endorsing surprising and suspicious convergence remains prevalent. Here are some provocative examples:

  1. The lack of path changes. Pace personal fit, friction, sunk capital, etc. it seems people who select careers on ‘non EA grounds’ often retain them after ‘becoming’ EA, and then provide reasons why (at least for them) persisting in their career is the best option.
  2. The claim that, even granting the overwhelming importance of the far future, it turns out that animal welfare charities are still the best to give to, given their robust benefits, positive flow through effects, and the speculativeness of far future causes.
  3. The claim that, even granting the overwhelming importance of the far future, it turns out that global poverty charities are still the best to give to, given their robust benefits, positive flow through effects, and the speculativeness of far future causes.
  4. Claims from enthusiasts of Cryonics or anti-aging research that this, additional to being good for their desires for an increased lifespan, is also a leading ‘EA’ buy.
  5. A claim on behalf of veganism that it is the best diet for animal welfare and for the environment and for individual health and for taste.

All share similar features: one has prior commitments to a particular cause area or action. One becomes aware of a new consideration which has considerable bearing on these priors. Yet these priors don’t change, and instead ancillary arguments emerge to fight a rearguard action on behalf of these prior commitments – that instead of adjusting these commitments in light of the new consideration, one aims to co-opt the consideration to the service of these prior commitments.

Naturally, that some rationalize doesn’t preclude others being reasonable, and the presence of suspicious patterns of belief doesn’t make them unwarranted. One may (for example) work in global poverty due to denying the case for the far future (via a person affecting view, among many other possibilities) or aver there are even stronger considerations in favour (perhaps an emphasis on moral uncertainty and peer disagreement and therefore counting the much stronger moral consensus around stopping tropical disease over (e.g.) doing research into AI risk as the decisive consideration).

Also, for weaker claims, convergence is much less surprising. Were one to say on behalf of veganism: “It is best for animal welfare, but also generally better for the environment and personal health than carnivorous diets. Granted, it does worse on taste, but it is clearly superior all things considered”, this seems much less suspect (and also much more true) than the claim it is best by all of these metrics. It would be surprising if the optimal diet for personal health did not include at least some animal products.

Caveats aside, though, these lines of argument are suspect, and further inspection deepens these suspicions. In sketch, one first points to some benefits the prior commitment has by the lights of the new consideration (e.g. promoting animal welfare promotes antispeciesism, which is likely to make the far future trajectory go better), and second remarks about how speculative searching directly on the new consideration is (e.g. it is very hard to work out what we can do now which will benefit the far future).6

That the argument tends to end here is suggestive of motivated stopping. For although the object level benefits of (say) global poverty are not speculative, their putative flow-through benefits on the far future are speculative. Yet work to show that this is nonetheless less speculative than efforts to ‘directly’ work on the far future is left undone.7 Similarly, even if it is the case the best way to make the far future go better is to push on a proxy indicator, which one? Work on why (e.g.) animal welfare is the strongest proxy out of competitors also tends to be left undone.8 As a further black mark, it is obviously suspect that those maintaining global poverty is the best proxy almost exclusively have prior commitments to global poverty causes, mutatis mutandis animal welfare, and so on.

We at least have some grasp of what features of (e.g.) animal welfare interventions make them good for the far future. If this (putatively) was the main value of animal welfare interventions due to the overwhelming importance of the far future, it would seem wise to try and pick interventions which maximize these features. So we come to a recursion: within animal welfare interventions, ‘object level’ and ‘far future’ benefits would be expected to only weakly converge. Yet (surprisingly and suspiciously) the animal welfare interventions recommended by the lights of the far future are usually the same as those recommended on ‘object level’ grounds.

Conclusion

If Oliver were biased, he would be far from alone. Most of us are (like it or not) at least somewhat partisan, and our convictions are in part motivated by extra-epistemic reasons: be it vested interests, maintaining certain relationships, group affiliations, etc. In pursuit of these ends we defend our beliefs against all considerations brought to bear against them. Few beliefs are indefatigable by the lights of any reasonable opinion, and few policy prescriptions are panaceas. Yet all of ours are.

It is unsurprising the same problems emerge within effective altruism: a particular case of ‘pretending to actually try’ is ‘pretending to take actually arguments seriously’.9 These problems seem prevalent across the entirety of EA: that I couldn’t come up with good examples for meta or far future cause areas is probably explained by either bias on my part or a selection effect: were these things less esoteric, they would err more often.10

There’s no easy ‘in house’ solution, but I repeat my recommendations to Eleanor: as a rule, maintaining dialogue, presuming good faith, engaging on the merits, and listening to others seems a better strategy, even if we think bias is endemic. It is also worth emphasizing the broad (albeit weak) convergence between cause areas is fertile common ground, and a promising area for moral trade. Although it is unlikely that the best thing by the lights of one cause area is the best thing by the lights of another, it is pretty likely it will be pretty good. Thus most activities by EAs in a particular field should carry broad approbation and support from those working in others.

I come before you a sinner too. I made exactly the same sorts of suspicious arguments myself on behalf of global poverty. I’m also fairly confident my decision to stay in medicine doesn’t really track the merits either – but I may well end up a beneficiary of moral luck. I’m loath to accuse particular individuals of making the mistakes I identify here. But, insofar as readers think this may apply to them, I urge them to think again.11

A subsequent problem is how to disentangle bias from expertise or privileged access. Oliver could suggest that those involved in the opera gain ‘insider knowledge’, and their epistemically superior position explains why they disproportionately claim the opera is best for human welfare.

Some features can help distinguish between bias and privileged access, between insider knowledge and insider beliefs. We might be able to look at related areas, and see if ‘insiders’ have superior performance which an insider knowledge account may predict (if insiders correctly anticipate movements in consensus, this is suggestive they have an edge). Another possibility is to look at migration of beliefs. If there is ‘cognitive tropism’, where better cognizers tend to move from the opera to AMF, this is evidence against donating to the opera in general and the claim of privileged access among opera-supporters in particular. Another is to look at ordering: if the population of those ‘exposed’ to the opera first and then considerations around human welfare are more likely to make Oliver’s claims than those exposed in reverse order, this is suggestive of bias on one side or the other.

It may well be that there are convincing answers to the object level questions, but I have struggled to find them. And, in honesty, I find the lack of public facing arguments in itself cause for suspicion.

(There are, of course, also tactical reasons that may speak against saying or doing very strange things.)

It seems to me that more people move from global poverty to far future causes than people move in the opposite direction (I suspect, but am less sure, the same applies between animal welfare and the far future). It also seems to me that (with many exceptions) far future EAs are generally better informed and cleverer than global poverty EAs.

I don’t have great confidence in this assessment, but suppose I am right. This could be adduced as evidence in favour of far future causes: if the balance of reason favoured the far future over global poverty, this would explain the unbalanced migration and ‘cognitive tropism’ between the cause areas.

But another plausible account explains this by selection. Global poverty causes are much more widely known that far future causes. Thus people who are ‘susceptible’ to be persuaded by far future causes were often previously persuaded by global poverty causes, whilst the reverse is not true – those susceptible to global poverty causes are unlikely to encounter far future causes first. Further, as far future causes are more esoteric, they will be disproportionately available to better-informed people. Thus, even if the balance of reason was against the far future, we would still see these trends and patterns of believers.

I am generally a fan of equal-weight views, and of being deferential to group or expert opinion. However, selection effects like these make deriving the balance of reason from the pattern of belief deeply perplexing.


  1.  We may wonder why this is the case: the content of the different moral theories are pretty alien to one another (compare universalizable imperatives, proper functioning, and pleasurable experiences). I suggest the mechanism is implicit selection by folk or ‘commonsense’ morality. Normative theories are evaluated at least in part by how well they accord to our common moral intuitions, and they lose plausibility commensurate to how much violence they do to them. Although cases where a particular normative theory apparently diverges from common sense morality are well discussed (consider Kantianism and the inquiring murder, or Utilitarianism and the backpacker), moral theories that routinely contravene our moral intuitions are non-starters, and thus those that survive to be seriously considered somewhat converge with common moral intuitions, and therefore one another. 
  2.  There may be some asymmetry: on the object level we may anticipate the ‘flow forward’ effects of global health on x-risk to be greater than the ‘flow back’ benefits of x-risk work on global poverty. However (I owe this to Carl Shulman) the object level benefits are probably much smaller than more symmetrical ‘second order’ benefits, like shared infrastructure, communication and cross-pollination, shared expertise on common issues (e.g. tax and giving, career advice). 
  3.  But not always. Some things are so hard to estimate directly, and using proxy measures can do better. The key question is whether the correlation between our outcome estimates and the true values is greater than that between outcome and (estimates of) proxy measure outcome. If so, one should use direct estimation; if not, then the proxy measure. There may also be opportunities to use both sources of information in a combined model. 
  4.  One example I owe to Stefan Shubert: we generally take the fact someone says something as evidence it is true. Pointing out relevant ‘ad hominem’ facts (like bias) may defeat this presumption. 
  5.  Population data – epistemic epidemiology, if you will – may help. If we find that people who were previously committed to the operas much more commonly end up claiming the opera is best for human welfare than than other groups, this is suggestive of bias. 
  6.  Although I restrict myself to ‘meta’-level concerns, I can’t help but suggest the ‘object level’ case for these things looks about as shaky as Oliver’s object level claims on behalf of the opera. In the same way we could question: “I grant that the arts is the an important aspect of human welfare, but is it the most important (compared to, say, avoiding preventable death and disability)?” or “What makes you so confident donations to the opera are the best for the arts – why not literature? or perhaps some less exoteric music?” We can post similarly tricky questions to proponents of 2-4: “I grant that (e.g.) antispeciesism is an important aspect of making the far future go well, but is it the most important aspect (compared to, say, extinction risks)?” or “What makes you so confident (e.g) cryonics is the best way of ensuring greater care for the future – what about militating for that directly? Or maybe philosophical research into whether this is the correct view in the first place?” 
  7.  At least, undone insofar as I have seen. I welcome correction in the comments. 
  8.  The only work I could find taking this sort of approach is this
  9.  There is a tension between ‘taking arguments seriously’ and ‘deferring to common sense’. Effective altruism only weakly converges with common sense morality, and thus we should expect their recommendations to diverge. On the other hand, that something lies far from common sense morality is a pro tanto reason to reject it. This is better acknowledged openly: “I think the best action by the lights of EA is to research wild animal suffering, but all things considered I will do something else, as how outlandish this is by common sense morality is a strong reason against it”. 
  10.  This ‘esoteric selection effect’ may also undermine social epistemological arguments between cause areas: 
  11.  Thanks to Carl Shulman, Stefan Schubert, Owen Cotton-Barratt, Amanda MacAskill and Pablo Stafforini for extensive feedback and advice. Their kind assistance should not be construed as endorsement of the content, nor responsibility for any errors within. 

At what cost, carnivory?

Summary: Using Animal Charity Evaluator’s figures, I estimate the amount donated to an effective animal charity to equal the harm caused by a typical American diet compared to veganism. This figure is surprisingly low: $2-5 per year. This suggests that personal dietary change, relative to other things we can do, is fairly ineffective. Yet most EAs interested in animal welfare are eager that others, including other EAs, stop using animal products. I explore a variety of means of resolving this tension, and recommend a large downward adjustment to the efficacy of animal charities is the best solution.[ref]This is a follow-on from an earlier attempt. The more involved discussion of non-consequentialist approaches I owe to comments on fb and the EA forum in general, and to Carl Shulman’s helpful remarks in particular.[/ref] Continue reading “At what cost, carnivory?”

Don’t sweat diet?

Summary: Various people (Hurford, Tomasik) have tried to estimate the typical animal welfare cost of a carnivorous diet. These costs have encouraged EAs to become vegetarians or vegans, and EAs particularly focused on animal welfare advocate that others (including EAs) should reduce their consumption of animal products.

Closely following an estimate by Kaufman, I consider the face value of abstaining from dairy or abstaining from all animal products in terms of donations to ACE-recommended animal welfare charities: <1 cent per year given to The Humane League would offset typical dairy consumption, and <$1 year would offset a typical American’s consumption of all animal products. Thus the emphasis on dietary change intra-EA seems misplaced: it is extraordinarily low impact compared to other means to helping animals.  Continue reading “Don’t sweat diet?”

The ethics (and economics) of paying doctors

If you have friends who are doctors (or keep a close eye on the national news) you will have heard of the recent bust up over a contract for junior doctors. The very short summary is the Department of health wanted to reform existing contracts for doctors, and started negotiations with the BMA. These broke down, but the government threatened to impose the contract anyway, junior doctors, in turn have threatened to strike. [ref] For much, much more detail, consider reading NHS Employers, the BMA, and the DDRB (renumeration board) proposals.[/ref]

I am much more ambivalent about the contract than most of my peers, who are varying shades of outraged. I’m not sure why. Continue reading “The ethics (and economics) of paying doctors”