Philip E. Tetlock on Forecasting and Foraging as a Fox (Ep. 93)
Today, I am speaking to Philip Tetlock, who, quite simply, is one of the greatest social scientists in the world.
今天，我要与菲利普·泰洛克交谈，他是世界上最伟大的社会科学家之一。
Up until now, we’ve been doing Conversations with Tyler face-to-face.
到目前为止，我们一直在与泰勒进行面对面的对话。
But for obvious reasons, Phillip is in Philadelphia.
但由于显而易见的原因，菲利普在费城。
He teaches at University of Pennsylvania.
他在宾夕法尼亚大学任教。
And I’m here in Arlington, Virginia.
我现在在弗吉尼亚州阿灵顿。
On forecasting
关于预测
Let’s just jump right into it.
让我们直接开始吧。
First question, Philip: with our forecasters, do we want accuracy or do we want them to be a kind of portfolio to make us more aware of extreme events and possibilities?
第一个问题，菲利普：我们的预测员是想要准确性，还是希望它们成为一种投资组合，让我们更加了解极端事件和可能性？
We want a lot of things from our forecasters, and accuracy is often not the first thing.
我们希望从预报员那里得到很多东西，而准确性往往不是首要的事情。
I think that we look to forecasters for ideological reassurance, we look to forecasters for entertainment, and we look to forecasters for minimizing regret functions of various sorts.
我认为，我们期待预测者能够提供意识形态上的保证，我们期待预测者能够带来娱乐，我们期待预测者能够最大限度地减少各种遗憾的功能。
We would really regret not having anticipated X, Y, or Z, so we want to pump up the probabilities of those things.
我们真的会后悔没有预见到 X、Y 或 Z，所以我们想提高这些事情发生的概率。
But if we take, say, the coronavirus — if we had had a few more extreme nuts who were maybe wrong most of the time but insisting that we needed to fear the next pandemic, wouldn’t we have been better off with that kind of portfolio, and thus, we don’t actually want more accuracy from our forecasters?
但是，如果我们以冠状病毒为例——如果我们有一些更极端的疯子，他们可能在大多数时候都是错误的，但坚持认为我们需要担心下一次大流行，那么我们拥有这种投资组合不是会更好吗？因此，我们实际上并不希望我们的预测者更加准确？
Well, in some sense we already did have that portfolio.
嗯，从某种意义上说，我们确实已经拥有了这个投资组合。
It was a mainstream position among epidemiologists for the last 20 years or so that you had a recipe for disaster.
过去 20 年左右，流行病学家的主流观点是，这就是灾难的根源。
David Epstein, the guy who recently wrote Range, a very interesting guy — you may have had him on your show, I don’t know — but he recently quoted himself from a 2007 newsletter that he wrote, and he said something like, “The presence of a large reservoir of SARS-like viruses among horseshoe bats, combined with a culture of eating exotic meat, is a time bomb.”
大卫·爱泼斯坦（David Epstein）是最近撰写《Range》的人，一个非常有趣的人 ——“你可能在你的节目中见过他，我不知道”——但他最近引用了自己在 2007 年写的时事通讯中的话，他说，“马蹄蝠中存在大量类似 SARS 的病毒，再加上吃异国肉类的文化，这是一颗定时炸弹。”
But what’s the problem —
但问题是什么——
That was in microbiology books.
那是微生物学书籍上的内容。
In the first half decade of the 21st century, it was common knowledge.
在21世纪上半叶，这是常识。
And indeed, even before SARS 1, even before the first SARS outbreak — some epidemiologists prefer to call COVID-19, SARS 2 — but even before SARS 1, epidemiologists were acutely aware of this, so it’s not as though we didn’t have it in our portfolio.
事实上，甚至在 SARS 1 之前，甚至在第一次 SARS 爆发之前 ——一些流行病学家更喜欢称 COVID-19、SARS 2 ——但即使在 SARS 1 之前，流行病学家就敏锐地意识到了这一点，所以我们的投资组合中并不是没有它。
We did.
我们做到了。
So those forecasters maybe weren’t entertaining enough?
那么那些预测者可能不够有趣？
Isn’t then, the margin we want to work on to make our better forecasters more entertaining and not more accurate.
那么，我们想要努力让我们更好的预测变得更有趣而不是更准确，这不是吗？
Yes, no?
是的，不是？
Well, the Signal-to-noise ratio isn’t going to be great.
嗯，信噪比不会很好。
I can assure you of that because there are plenty of people who are naturally more entertaining than epidemiologists.
我可以向你保证这一点，因为有很多人天生比流行病学家更有趣。
But maybe the whole portfolio needs to be more vivid rather than trying to fine-tune the accuracy of particular parts of it.
但也许整个作品集需要更加生动，而不是试图微调其中特定部分的准确性。
Right?
正确的？
We have lots of people competing in the marketplace of ideas for attention, and that’s a hard competition for scientists to beat.
我们有很多人在思想市场上争夺注意力，这对科学家来说是一场艰难的竞争。
What do you think of the argument that science only exists at all Because most scientists are overconfident that if they were rational Bayesians, they would just latch onto the opinions of the smartest and best-trained people before them.
你如何看待科学只存在的论点？因为大多数科学家都过于自信，认为如果他们是理性的贝叶斯主义者，他们就会抓住他们之前最聪明和训练有素的人的观点。
That there’s only progress precisely because people are making forecasting mistakes.
正是因为人们犯了预测错误，才会取得进步。
Right.
正确的。
you’ll hear versions to that argument, I suppose, on Wall Street as well.
我想，在华尔街你也会听到这种说法的不同版本。
Sure.
当然。
There’s a great deal of truth to it.
这其中有很多道理。
I certainly have been guilty of overconfidence at many junctures in my career, thinking I’m going to be able to take on things that looked impossible and often turned out to be impossible.
当然，在我职业生涯的许多关键时刻，我都因过度自信而感到内疚，认为我能够承担那些看起来不可能、而且往往被证明是不可能的事情。
Most projects that most scientists embark on, I think, don’t succeed.
我认为，大多数科学家开展的大多数项目都没有成功。
It does take a certain amount of quasi-irrational persistence.
它确实需要一定程度的准非理性的坚持。
As with Columbus, right?
就像哥伦布一样，对吗？
Or the founding of the United States arguably was irrational to break away from the British Empire, right?
或者美国建国脱离大英帝国可以说是非理性的，对吧？
Which was doing pretty well back then.
当时表现相当不错。
It seemed like a risk-seeking move.
这似乎是一个寻求风险的举动。
But say I set up an alternate research program, and I sought to take forecasters and (A) make them more entertaining, and (B) maybe I’d give them uppers, so they were more overconfident.
但是假设我设立了一个替代研究项目，我试图让预测者（A）让他们更有趣，（B）也许我会给他们上层，所以他们更加过度自信。
Would that do the world good?
这会给世界带来好处吗？
Well, you could certainly have induced the epidemiologists who are worried about the horseshoe bats in central China or other possible sources of zoonotic viruses.
好吧，你肯定会引起流行病学家对中国中部的马蹄蝠或其他可能的人畜共患病毒来源的担忧。
You could certainly have induced them to pump up their probabilities.
你当然可以诱导他们提高概率。
And you would, of course, have started to run into a problem of crying wolf.
当然，你会开始遇到狼来了的问题。
They’d been saying there’s a 30, 40, 50 percent chance of a viral leap into human beings each year, and it didn’t happen.
他们一直说每年有 30%、40%、50% 的可能性病毒会传染给人类，但这并没有发生。
Didn’t happen.
没有发生。
Didn’t happen.
没有发生。
You get a crying-wolf effect, right?
你会得到“狼来了”的效果，对吗？
On financial markets and super predictors
关于金融市场和超级预测者
Right.
对。
How do you think about financial markets in relation to your work on super predictors?
您如何看待金融市场与您的超级预测器工作之间的关系？
Are financial markets, in essence, super predictors to begin with?
金融市场本质上是超级预测者吗？
Or can super predictors, on average, beat financial markets?
或者超级预测者平均能够击败金融市场吗？
Oh, boy!
哦，孩子！
We play with prediction markets with the work with the intelligence community, going all the way back to Admiral Poindexter and the original DARPA effort to launch prediction markets inside the intelligence community.
我们通过与情报界的合作来研究预测市场，可以追溯到波因德克斯特海军上将和 DARPA 最初在情报界内部推出预测市场的努力。
I think your colleague, Robin Hanson, was involved with some of that work around 2000.
我认为你的同事 Robin Hanson 在 2000 年左右参与了其中的一些工作。
I would say around 2003, 2004 — in that range.
我想说的是 2003 年、2004 年左右 ——在这个范围内。
We’ve also been working with the prediction markets in parallel with forecasting tournaments, and there are pros and cons to each method of eliciting judgments.
我们在预测锦标赛的同时也一直在与预测市场合作，每种引发判断的方法都各有利弊。
But the IC, the intelligence community, doesn’t let us make those markets deep and liquid the way they are on Wall Street.
但情报界不允许我们像华尔街那样让这些市场变得深度和流动。
People are essentially competing for reputational points the way they are in forecasting tournaments, and monetary prizes are either small or nonexistent.
人们本质上是在争夺声誉积分，就像预测锦标赛一样，而金钱奖励要么很小，要么根本不存在。
So most economists, I think, would not consider that to be a very robust test of the efficacy of prediction markets.
因此，我认为大多数经济学家不会认为这是对预测市场有效性的非常有力的测试。
And when prediction markets fall short — and they do typically not perform quite as well as forecasting tournaments — when they fall short, it’s hardly a decisive rebuke of the market mechanism for eliciting forecast.
当预测市场达不到要求时——而且它们的表现通常不如预测锦标赛——当它们达不到要求时，这很难说是对引发预测的市场机制的决定性谴责。
There are lots of very powerful institutional actors like Goldman Sachs and so forth that are continually trying to do exactly what you describe.
有很多非常强大的机构参与者，如高盛等，他们不断地尝试做你所描述的事情。
It’s an ongoing process.
这是一个持续的过程。
You’re an economist.
你是一位经济学家。
I don’t need to tell you that.
我不需要告诉你这一点。
Sure.
当然。
There are implicit prediction markets in, say, coronavirus, say, prices of airline stocks, right?
比如说，冠状病毒、航空公司股票的价格都存在隐含的预测市场，对吗？
And those are very liquid.
这些都是非常流动的。
They’ve been very thickly traded lately.
最近它们的交易量非常大。
They have indeed.
他们确实有。
If you took your 10 best superforecasters and brought them into the hedge fund people at Goldman Sachs and you all sat down together, who would be teaching whom?
如果你把 10 位最好的超级预测家带入高盛的对冲基金人员，然后大家坐在一起，谁会教谁？
It’s an interesting experiment.
这是一个有趣的实验。
My project manager from the first set of forecasting tournaments, Terry Murray, founded a company, Good Judgment Incorporated, which does things like that.
我的第一组预测锦标赛项目经理 Terry Murray 创立了一家名为 Good Judgment Incorporated 的公司，专门做类似的事情。
So that’s a proprietary venture.
所以这是一个专有企业。
You’d probably want to talk to Terry about how successful or not successful they’ve been in doing that.
你可能想和特里谈谈他们在这方面做得有多成功或不成功。
I think they’ve had some success.
我认为他们取得了一些成功。
It’s extremely hard to do that.
做到这一点非常困难。
It’s nontrivial.
这很重要。
I think there’s a good deal of similarity in the cognitive ability, cognitive-style profiles of superforecasters and the kinds of people you see on the staffs of Goldman Sachs.
我认为超级预测家的认知能力、认知风格特征和高盛员工的类型有很多相似之处。
It would be a tight race.
这将是一场激烈的比赛。
What about the sports betting market, do you think there are inefficiencies in that because they don’t have enough superforecasters?
体育博彩市场怎么样，您认为由于没有足够的超级预测员而导致效率低下吗？
I’m not an expert on sports betting.
我不是体育博彩专家。
Better to talk to Nate Silver about that.
最好和内特·西尔弗谈谈这件事。
Let me put the question more generally.
让我更笼统地提出这个问题。
There are many markets out there which predict something.
有很多市场可以预测一些事情。
Sports betting markets are simply the most obviously, most explicit about prediction.
体育博彩市场是最明显、最明确的预测。
So if there was something the world didn’t know about prediction already, those markets should be inefficient.
因此，如果世界尚不知道预测的某些内容，那么这些市场应该是低效的。
Yes or no?
是还是不是？
Why would you think the answer would be yes?
为什么你认为答案是肯定的？
Well, let’s say that people favored the home team too much.
好吧，可以说人们太偏爱主队了。
Too many people might bet on the New York teams, the Los Angeles teams, and then the odds would be skewed.
太多人可能会押注于纽约队、洛杉矶队，那么赔率就会出现偏差。
So if there’s a bias in people without superforecasting techniques, we would expect sports odds to somehow be off.
因此，如果没有超级预测技术的人存在偏见，我们预计体育赔率会以某种方式下降。
Or at least they would have been off before your work was published.
或者至少在你的作品发表之前他们就已经关闭了。
Well, our arbitrage predated superforecasting.
嗯，我们的套利早于超级预测。
But isn’t arbitrage itself superforecasting?
但套利本身不就是超级预测吗？
People are arbitraging —
人们在套利——
Yes, in a form.
是的，以某种形式。
— on the basis of some set of information.
— 基于一些信息。
I think that’s fair, yeah.
我认为这很公平，是的。
So all these markets out there — do you think they are, without you, already the best available superforecasters?
那么，所有这些市场 — 如果没有你，你是否认为它们已经是最好的超级预测者了？
The term best is a term I’m just not comfortable with.
“最好”这个词我只是觉得不舒服。
I rather doubt that they’re at the optimal forecasting frontier at the moment.
我相当怀疑他们目前是否处于最佳预测前沿。
But they’re often probably fairly close.
但它们通常可能相当接近。
And is there room to incentivize people how to predict the market?
是否有空间激励人们如何预测市场？
Well, that’s one of those paradoxes that economists have written about, right?
嗯，这是经济学家所写的悖论之一，对吧？
When people predict out to many decimal places, do you think that’s absurd?
当人们预测到小数点后很多位时，你认为这很荒谬吗？
Or do you think it’s useful?
或者说你觉得有用吗？
What’s the optimal level of granularity for different categories of forecasts?
不同类别的预测的最佳粒度级别是多少？
For the kinds of things we were looking at in the IARPA original forecasting tournaments with geopolitical events, like how long the Syrian Civil War would last, or what would Russia do in Eastern Ukraine, or things of that sort — yes, it would be absurd to go to three or four decimals points.
对于我们在 IARPA 最初的地缘政治事件预测比赛中看到的事情，比如叙利亚内战会持续多久，或者俄罗斯会在乌克兰东部做什么，或者类似的事情 —— 是的，精确到小数点后三到四位是荒谬的。
The National Intelligence Council, which synthesizes a lot of intelligence analysis — originally, they only distinguished five degrees of uncertainty, and they didn’t put numbers on it.
国家情报委员会综合了很多情报分析——本来他们只区分了五个程度的不确定性，并没有给出数字。
More recently, they’ve moved to seven degrees of uncertainty, and they do put numerical ranges on, so “somewhat likely” represents a certain probability range.
最近，他们已经将不确定性提高到了七度，并且确实给出了数值范围，因此“有些可能”代表了一定的概率范围。
Now, in our work, we’ve explored how granular the best forecasters are.
现在，在我们的工作中，我们已经探索了最佳预测者的精细程度。
Doing various rounding experiments, we round a forecast off to the nearest tenth, that kind of thing.
通过进行各种舍入实验，我们将预测四舍五入到最接近的十分之一，诸如此类。
If it’s pseudo precision — if when they adjust it and they move from 0.6 to 0.65, for example — if on average, that doesn’t improve their accuracy, we would conclude that they can’t achieve that level of granularity.
如果是伪精度——例如，当他们调整它并从 0.6 移动到 0.65 时——如果平均而言，这并不能提高他们的准确性，我们就会得出结论，他们无法达到那种粒度水平。
Our best statistical estimates are that forecasters, for the types of questions the intelligence community often poses, can distinguish between 10 and 15 degrees of uncertainty, which is considerably more than the seven they think they can now, a lot more than the five they thought they used to be able to distinguish.
我们最好的统计估计是，对于情报界经常提出的问题类型，预测者可以区分 10 到 15 度的不确定性，这比他们认为现在可以区分的 7 度要高得多，也比他们认为过去能够区分的 5 度高得多。
But how useful it is to be able to distinguish varying degrees of uncertainty is going to hinge on the kind of game you’re playing.
但区分不同程度的不确定性的有用程度取决于你正在玩的游戏类型。
If it’s poker, you might well want someone who’s very adept at distinguishing things to, say, three decimal points.
如果是扑克，你很可能需要一个非常擅长区分事物的人，比如小数点后三位。
If you could take just a bit of time away from your research and play in your own tournaments, are you as good as your own best superforecasters?
如果您可以花一点时间从研究中抽出一点时间来参加自己的锦标赛，您是否和自己最好的超级预测家一样出色？
I don’t think so.
我不这么认为。
I don’t think I have the patience or the temperament for doing it.
我认为我没有耐心或脾气去做这件事。
I did give it a try in the second year of the first set of forecasting tournaments back in 2012, and I monitored the aggregates.
我确实在 2012 年第一组预测锦标赛的第二年进行了尝试，并监控了总计。
We had an aggregation algorithm that was performing very well at the time, and it was outperforming 99.8 percent of the forecasters from whom the composite was derived.
我们有一个当时表现非常好的聚合算法，它的表现优于 99.8% 的综合预测者。
If I simply had predicted what the composite said at each point in time in that tournament, I would have been a super superforecaster.
如果我简单地预测了比赛中每个时间点的综合数据，我就会成为一个超级超级预测者。
I would have been better than 99.8 percent of the superforecasters.
我会比 99.8% 的超级预测者更好。
So, even though I knew that it was unlikely that I could outperform the composite, I did research some questions where I thought the composite was excessively aggressive, and I tried to second guess it.
因此，尽管我知道我不太可能胜过复合材料，但我确实研究了一些我认为复合材料过于激进的问题，并试图重新猜测它。
The net result of my efforts — instead of finishing in the top 0.02 percent or whatever, I think I finished in the middle of the superforecaster pack.
我努力的最终结果 —— 我认为我并没有排在前 0.02% 之类的位置，而是排在了超级预测者的中间位置。
That doesn’t mean I’m a superforecaster.
这并不意味着我是超级预测家。
It just means that when I tried to make a forecast better than the composite, I degraded the accuracy significantly.
这只是意味着，当我试图做出比综合预测更好的预测时，我显着降低了准确性。
But what do you think is the kind of patience you’re lacking?
但你认为你缺乏的耐心是什么？
Because if I look at your career, you’ve been working on these databases on this topic for what?
因为如果我看看你的职业生涯，你一直在研究这个主题的这些数据库是为了什么？
Over 30 years.
30多年了。
That’s incredible patience, right?
这真是令人难以置信的耐心，对吧？
More patience than most of your superforecasters have shown.
比大多数超级预测家表现出的耐心还要多。
Is there some dis-aggregated notion of patience where they have it and you don’t?
是否存在某种分解的耐心概念，他们有，而你没有？
Yeah, they have a skill set.
是的，他们有一套技能。
In the most recent tournaments, we’ve been working on with them, this becomes even more evident — their willingness to delve into the details of really pretty obscure problems for very minimal compensation is quite extraordinary.
在最近的比赛中，我们一直在与他们合作，这一点变得更加明显——他们愿意以极低的报酬深入研究真正相当晦涩的问题的细节，这是非常非凡的。
They are intrinsically cognitively motivated in a way that is quite remarkable.
他们本质上具有认知动机，其方式非常引人注目。
How am I different from that?
我与那有何不同？
I guess I have a little bit of attention deficit disorder, and my attention tends to roam.
我想我有一点注意力缺陷障碍，而且我的注意力容易走神。
I’ve not just worked on forecasting tournaments.
我不仅仅致力于预测比赛。
I’ve been fairly persistent in pursuing this topic since the mid 1980s.
自 20 世纪 80 年代中期以来，我一直相当坚持地研究这个主题。
Even before Gorbachev became general party secretary, I was doing a little bit of this.
甚至在戈尔巴乔夫担任党总书记之前，我就在做一些这样的事情。
But I’ve been doing a lot of other things as well on the side.
但我同时也做了很多其他事情。
My attention tends to roam.
我的注意力倾向于漫游。
I’m interested in taboo tradeoffs.
我对禁忌权衡感兴趣。
I’m interested in accountability.
我对责任感很感兴趣。
There’re various things I’ve studied that don’t quite fall in this rubric.
我研究过的很多东西并不完全属于这个标题。
Doesn’t that make you more of a fox though?
这难道不会让你更像一只狐狸吗？
You know something about many different areas.
您对许多不同领域有所了解。
I could ask you about antebellum American discourse before the Civil War, and you would know who had the smart arguments and who didn’t.
我可以向你询问内战前美国内战前的言论，你就会知道谁有聪明的论点，谁没有。
Right?
正确的？
Well, I would know who has arguments to take the more integratively complex forms — on the one hand, on the other hand, and then synthesis.
好吧，我会知道谁有理由采取更综合复杂的形式——一方面，另一方面，然后综合。
Whether they want to consider those arguments smarter or not is another matter.
他们是否想更明智地考虑这些论点是另一回事。
But yes, I suppose that’s fair.
但是，是的，我认为这是公平的。
I’ve always resonated a little more to the foxes than to the hedgehogs, but I think when you look at the great achievements in science, they often come from hedgehogs.
我总是对狐狸产生更多的共鸣，而不是对刺猬，但我认为当你看到科学上的伟大成就时，它们往往来自刺猬。
On arguments about COVID-19
关于 COVID-19 的争论
If you look today at the ongoing debates about coronavirus and what will happen — and I mean now the debates amongst the smart people, the people you respect — what is the mistake you see them making, the biggest mistake?
如果你看看今天正在进行的关于冠状病毒以及将会发生什么的辩论——我指的是聪明人之间的辩论，你尊敬的人——你看到他们犯的错误是什么，最大的错误是什么？
Oh, you want me to be an amateur epidemiologist here.
哦，你想让我成为这里的一名业余流行病学家。
No, mistaken reasoning.
不，推理错误。
You don’t have to give your numerical estimate.
您不必给出您的数值估计。
Procedurally, what are they not getting right?
从程序上看，他们有哪些地方做得不对？
Well, is it a mistake if you’re a public health expert who feels that one mistake is much worse than the other.
好吧，如果你是一名公共卫生专家，觉得一个错误比另一个错误严重得多，那么这是一个错误吗？
It’s much better to overestimate the threat of the virus than to underestimate it because you have to influence public opinion and public behavior.
高估病毒的威胁比低估病毒的威胁要好得多，因为你必须影响公众舆论和公众行为。
There, you’re not forecasting.
在那里，你不是在预测。
You’re engaged in manipulation, social influence.
你正在参与操纵、社会影响。
It comes back to your original question about what do we want from our forecasters?
这又回到了你最初的问题：我们希望从预测者那里得到什么？
And accuracy is only one of the things we want from them.
准确性只是我们希望从他们那里得到的东西之一。
We look to forecasters for a lot of things, be it to inspire confidence, inspire fear, and so forth.
我们向预测者寻求很多帮助，无论是激发信心、激发恐惧等等。
If we’re trying to estimate how much people cut back on their risk-taking behavior because they’re afraid of the virus, are the group of people best suited to do that epidemiologists, other social scientists, or your superforecasters?
如果我们试图估计有多少人因为害怕病毒而减少了冒险行为，那么最适合这样做的人是流行病学家、其他社会科学家还是你的超级预测者？
Maybe even economists, right?
也许甚至是经济学家，对吧？
We study elasticities.
我们研究弹性。
Why should it be the epidemiologists?
为什么应该是流行病学家？
I think you’d want an interdisciplinary team.
我认为你需要一个跨学科的团队。
Diversity is one of these words that’s been reduced to a cliché, but we have found in our work that cognitive diversity helps, and it helps in certain quite well-defined ways.
多样性是已经沦为陈词滥调的词之一，但我们在工作中发现认知多样性有所帮助，而且它以某些明确定义的方式有所帮助。
If you want to create a composite that out-predicts the vast majority of the superforecasters, a good way to do it is not only to take the most recent forecast of the best forecasters in the domain, but it’s also to extremize that forecast to the degree that people who normally disagree, agree with each other.
如果你想创建一个比绝大多数超级预测者预测更准确的综合结果，一个好方法不仅是采用该领域最佳预测者的最新预测，而且还要将该预测极端化到通常不同意的人们彼此同意的程度。
And when you have convergence among diverse observers, that’s a signal that the weighted average composite is probably too conservative and you should extremize.
当不同观察者之间出现趋同时，这表明加权平均综合数据可能过于保守，你应该采取极端措施。
Does the diverse team have a CEO, someone in charge?
多元化团队是否有首席执行官或负责人？
Not in this case, no. That’s done purely statistically.
不是在这种情况下，不是。这纯粹是统计上的结果。
But would you put someone in charge?
但你会派人来负责吗？
And maybe the person in charge would implement that statistical algorithm, right?
也许负责人会实施那个统计算法，对吧？
But who is the person you would put in charge of the team?
但你会任命谁来负责这个团队呢？
An epidemiologist?
流行病学家？
Yourself?
你自己？
Your best superforecaster?
你最好的超级预测家？
Bill Gates?
比尔·盖茨？
That’s a matter of managerial skill, I would say.
我想说，这是管理技巧的问题。
Going back to one of my old dissertation advisors at Yale, 40 plus years ago, Irv Janis on Groupthink, I would pick a leader who knows how to shut up, and not reveal opinions at the beginning of the meeting, and knows how to listen.
回到 40 多年前我在耶鲁大学的一位旧论文导师，Irv Janis 在 Groupthink 上的观点，我会选择一位懂得如何闭嘴、不在会议开始时透露意见、懂得如何倾听的领导者。
I would pick a leader who knows how to shut up, and not reveal opinions at the beginning of the meeting, and knows how to listen.
我会选择一位懂得如何闭嘴、不在会议开始时透露意见、懂得如何倾听的领导者。
And which group of people do you think that best describes?
您认为这最能描述哪一群人？
I think a lot of good executives have the intuition that you get more out of a team of forecasters or problem solvers if you elicit independent judgments initially that are uncontaminated by conformity pressure, and then you create an environment in which ideas can be freely critiqued before lifting the veil of anonymity and letting people see who’s taking which positions.
我认为很多优秀的高管都有这样的直觉：如果你一开始就能够引发不受整合压力影响的独立判断，那么你就能从预测者或问题解决者团队中获得更多收益，然后创造一个可以自由批评想法的环境，然后揭开匿名的面纱，让人们看到谁在采取什么立场。
Do you think having machine learning and artificial intelligence has made us much better at forecasting things right now?
您认为机器学习和人工智能是否让我们现在能够更好地预测事物？
Social events —
社交活动——
Certain categories of things for sure.
某些类别的事物是肯定的。
Sure, at the micro level, but social events: Whether there’ll be a recession.
当然，在微观层面，但社会事件：是否会出现经济衰退。
How many people will die from the coronavirus?
有多少人会死于冠状病毒？
Will we settle Mars?
我们会定居火星吗？
IARPA just ran a forecasting tournament called Hybrid Forecasting Competition in which they pitted algorithmic approaches and human approaches and hybrid approaches against each other.
IARPA 刚刚举办了一场名为混合预测竞赛的预测锦标赛，在比赛中，他们将算法方法、人类方法和混合方法相互竞争。
I should let IARPA speak for itself about how well its programs work or don’t work.
我应该让 IARPA 自己来说明它的项目是否有效。
I don’t think there’s a lot of evidence to support the claim that machine intelligence is well equipped to take on the sorts of problems that the intelligence community wanted to have answered when it runs the forecasting tournaments it’s been running with our research team.
我认为没有很多证据支持这样的说法，即机器智能有能力解决情报界在与我们的研究团队一起举办的预测锦标赛时想要回答的各种问题。
The things like the Syrian Civil War, Russia/Ukraine, settlement on Mars — these are events for which base rates are elusive.
像叙利亚内战、俄罗斯/乌克兰、火星定居这样的事件——这些事件的基本比率是难以捉摸的。
It’s not like you’re screening credit card applicants for Visa.
这不像你在筛选 Visa 信用卡申请人。
Machine intelligence is going to dominate human intelligence totally.
机器智能将完全主宰人类智能。
Machine intelligence dominates humans in Go and Chess.
机器智能在围棋和国际象棋中统治着人类。
It may not dominate humans in poker.
它可能不会在扑克中主宰人类。
I don’t know that the state of the art is quite there yet, but it’s in their StarCraft or whatever the next thing that Dennis Hassabis is going to conquer.
我不知道最先进的技术是否已经成熟，但这是在他们的《星际争霸》或丹尼斯·哈萨比斯接下来要征服的任何事物中。
No, I don’t see evidence that those approaches work in the domains that we study with the intelligence community.
不，我没有看到证据表明这些方法在我们与情报界研究的领域有效。
Do you think the hybrid man-machine approaches are overrated?
您认为混合人机方法是否被高估了？
No, it’s very domain specific.
不，它是非常特定于领域的。
It sounds like a great idea.
这听起来是个好主意。
Who could be against it?
谁会反对呢？
But the devil lurks in the details, and it doesn’t deliver as automatically as you might hope.
但魔鬼潜伏在细节中，它不会像你希望的那样自动实现。
It’s trench warfare here.
这里是堑壕战。
Do you think the world as a whole is becoming easier to predict or harder to predict?
您认为整个世界变得更容易预测还是更难预测？
And again, I mean social events.
再说一次，我指的是社交活动。
I’m not sure there’s a definite trend one way or the other.
我不确定是否存在某种明确的趋势。
You hear a lot of talk, a lot of claims that there are, but if you look back on the 20th century, there certainly were lots of major pockets of unpredictability.
你会听到很多说法，很多说法，但如果你回顾 20 世纪，肯定存在很多不可预测的重大领域。
It’s not clear.
目前还不清楚。
What if I say — again, current events aside — but it seems easier to predict.
如果我说 — 再说一遍，抛开当前事件 — 但它似乎更容易预测。
There hasn’t been a world war since 1945.
自 1945 年以来就没有发生过世界大战。
There’s been steady economic growth in most parts of the world.
世界大部分地区的经济持续稳定增长。
More peace.
更多平安。
Isn’t that easier to predict?
这样不是更容易预测吗？
You just predict 2 to 4 percent global economic growth and you pick up a fair amount of what’s happened since 1950.
你只要预测全球经济增长 2% 到 4%，你就会了解到 1950 年以来发生的很多事情。
Indeed.
确实如此。
So, simple extrapolation algorithms, historically, are hard to beat.
因此，从历史上看，简单的外推算法很难被击败。
We’re running a COVID-19 mini forecasting tournament right now, and they’re proving to be hard to beat.
我们现在正在举办一场 COVID-19 迷你预测锦标赛，事实证明他们很难被击败。
The skill, of course, is when to alter the trend, whether to accelerate it or decelerate it, or to change direction.
当然，技巧在于何时改变趋势，是加速还是减速，或者改变方向。
And do you have a personal intuition on that?
您对此有个人直觉吗？
I think humans have been repeatedly humbled in competitions against simple statistical algorithms, going back to Paul Meehl’s famous little book on clinical versus actuarial approaches to predicting in medicine and psychiatry.
我认为人类在与简单统计算法的竞争中一再被羞辱，这可以追溯到保罗·米尔（Paul Meehl）著名的小书，该书关于医学和精神病学中的临床与精算预测方法。
So, I would say, “Be humble.”
所以，我会说：“要谦虚。”
There’s some of your early research that, if I read it properly, suggests that making people accountable leads to more evasion and self-deception on their part.
你的一些早期研究表明，如果我正确地阅读它，让人们承担责任会导致他们更多的逃避和自欺欺人。
Are you worried that your work with pundits, by trying to make them more accountable, will lead to more evasion and self-deception from them?
您是否担心您与专家的合作，试图让他们更加负责任，会导致他们更多的逃避和自欺欺人？
How do you square early Tetlock and mid-period-to-late Tetlock?
如何调整早期 Tetlock 和中晚期 Tetlock 的关系？
It’s actually not too difficult in that particular case.
在这种特殊情况下，这实际上并不太困难。
It really depends on the type of accountability.
这实际上取决于责任的类型。
Tournaments create a very stark monistic type of accountability in which one thing and only one thing matters, and that is accuracy.
锦标赛创造了一种非常鲜明的一元问责制，其中只有一件事很重要，那就是准确性。
You get no points for being an ideological cheerleader and pumping up the probabilities of things that your team wants to be true or downplaying the probabilities of the things your team doesn’t want to be true.
如果你成为意识形态的拉拉队长，提高你的团队希望成为事实的事情的概率，或者淡化你的团队不希望成为事实的事情的可能性，那么你就得不到任何积分。
You take a reputational hit.
你的名誉会受到打击。
So the incentives are very unusually tightly aligned to favor accuracy.
因此，激励措施异常紧密地结合在一起，以支持准确性。
That’s extremely unusual in the social world.
这在社交界是极其不寻常的。
Most forms of accountability occur in organizational settings in which there are lots of distortions at work.
大多数形式的问责发生在工作中存在很多扭曲的组织环境中。
And the rational political response for a decision maker located in most accountability matrices and organizations is to engage in some mixture of strategic attitudes shifting toward the views of important others or, as you put it, evasion, procrastination, and so forth.
对于大多数问责矩阵和组织中的决策者来说，理性的政治反应是采取某种混合的战略态度，转向重要其他人的观点，或者正如你所说的，逃避、拖延等等。
But given that, when a pundit is on, say, the evening news, there’s not a little box at the bottom that gives the Tetlock score of that pundit, correct?
但考虑到这一点，当一位专家出现在晚间新闻中时，底部没有一个小方框给出该专家的泰洛克分数，对吗？
So most people don’t know the actual record.
所以大多数人并不知道真实的记录。
Given that, out there somewhere is a measure of how good or bad the pundit is, are you worried that, in a sense, you will make those pundits run further away from objective standards precisely because they do poorly by them?
考虑到，在某个地方有一个衡量专家好坏的标准，你是否担心，从某种意义上说，你会让那些专家远离客观标准，正是因为他们在客观标准上做得不好？
I don’t know if we can make them run much further away from objective standards than they already are.
我不知道我们是否能让他们比现在更加远离客观标准。
But that’s a very interesting point about forecasting tournaments.
但这是预测锦标赛的一个非常有趣的点。
I look at the kinds of people who are attracted to participate in them.
我会观察哪些人会被吸引来参与其中。
At the very outset, I invited lots of big shots to participate in forecasting tournaments, and they’ve repeatedly turned me down.
一开始我邀请了很多大人物参加预测比赛，但都被他们多次拒绝。
I had a very interesting correspondence with William Safire in the 1980s about forecasting tournaments.
在 20 世纪 80 年代，我与 William Safire 有过一次关于预测锦标赛的非常有趣的通信。
We could talk a little about it later.
我们可以稍后再讨论。
The upshot of this is that young people who are upwardly mobile see forecasting tournaments as an opportunity to rise.
这样做的结果是，那些具有向上流动性的年轻人将预测锦标赛视为一个上升的机会。
Old people like me and aging baby-boomer types who occupy relatively high status inside organizations see forecasting tournaments as a way to lose.
像我这样的老年人和在组织内占据相对较高地位的婴儿潮一代人将预测锦标赛视为一种失败的方式。
If I’m a senior analyst inside an intelligence agency, and say I’m on the National Intelligence Council, and I’m an expert on China and the go-to guy for the president on China, and some upstart R&D operation called IARPA says, “Hey, we’re going to run these forecasting tournaments in which we assess how well the analytic community can put probabilities on what Xi Jinping is going to do next.”
如果我是情报机构内部的高级分析师，并说我是国家情报委员会的成员，是中国问题专家，也是总统在中国问题上的得力助手，某个名为 IARPA 的新兴研发机构会说：“嘿，我们将举办这些预测比赛，评估分析界对习近平下一步行动的可能性的预测能力。”
And I’ll be on a level playing field, competing against 25-year-olds, and I’m a 65-year-old, how am I likely to react to this proposal, to this new method of doing business?
我将在一个公平的竞争环境中与 25 岁的人竞争，而我已经 65 岁了，我会对这个提议、这种新的经营方式做出什么反应？
It doesn’t take a lot of empathy or bureaucratic imagination to suppose I’m going to try to nix this thing.
不需要太多的同理心或官僚想象力就可以假设我会尝试阻止这件事。
Which nation’s government in the world do you think listens to you the most?
您认为世界上哪个国家的政府最听您的意见？
You may not know, right?
你可能不知道，对吧？
I might actually.
我可能真的。
I’m not going to say.
我不会说。
I suppose that the most prominent political fan I have at the moment is probably one of the senior advisors to Boris Johnson, Dominic Cummings, who recently caused a bit of a stir in the UK by appointing a superforecaster who had written some blogs that people interpreted as misogynist or racist or fascist or eugenicist or I don’t know, some mixture of all those things.
我想我目前最杰出的政治粉丝可能是鲍里斯·约翰逊（Boris Johnson）的高级顾问多米尼克·卡明斯（Dominic Cummings）之一，他最近任命了一位超级预测员，在英国引起了一些轰动，这位超级预测员写了一些博客，人们将这些博客解读为厌恶女性主义者、种族主义者、法西斯主义者、优生主义者，或者我不知道，是所有这些东西的混合体。
I think his name was Andrew Sabisky.
我认为他的名字是安德鲁·萨比斯基。
He was a young man, 24, 25, the kind of person .
他是个年轻人，24岁、25岁，就是这样的人。
.
。
.
。
young people are attracted, as I said before, to forecasting tournaments.
正如我之前所说，年轻人被预测比赛所吸引。
It’s a fast track toward upward mobility.
这是一条向上流动的快车道。
You have all the high-status people making vague verbiage forecasts, people like me.
所有地位高的人都在做出模糊的预测，比如我。
But do you think Cummings actually is influenced by you?
但你认为卡明斯实际上受到你的影响吗？
Because as I understand what he’s doing, correctly or not, he thinks he knows a bunch of things that other people do not.
因为据我了解他在做什么，无论正确与否，他认为他知道很多其他人不知道的事情。
And that seems somewhat non-Tetlockian, right?
这似乎有些非泰特洛克式的，对吧？
Maybe, for part of his portfolio of ideological armor, maybe he’s actually very non-Tetlockian, and it’s the people in Singapore who are your true fans.
也许，就他的意识形态盔甲组合的一部分而言，也许他实际上非常非泰特洛克式，而新加坡人民才是你真正的粉丝。
Well, there are some people in Singapore, too.
嗯，新加坡也有一些人。
It’s interesting you should mention that.
有趣的是你应该提到这一点。
Interesting you bring that one up.
有趣的是你提出了这个。
Mr. Cummings and Mr. Gove both separately brought up my work at various points during the Brexit debate.
卡明斯先生和戈夫先生在英国脱欧辩论期间的不同时刻分别提出了我的工作。
And Michael Gove, at least in the UK, famously said that Britain has had enough of experts.
迈克尔·戈夫（至少在英国）有句名言，英国已经受够了专家。
I don’t know if you remember that particular quote.
我不知道你是否还记得那句话。
Of course.
当然。
He evoked a support at least for that position, Expert Political Judgment, in which portions of that book compare subject matter experts to minimalist statistical baselines like extrapolation.
他至少唤起了对这一立场的支持，《专家政治判断》，其中该书的部分内容将主题专家与外推等最低限度的统计基线进行了比较。
Can you predict simple extrapolation algorithms?
你能预测简单的外推算法吗？
And the answer was often no. Gove was raising the point that, where do these guys get off making these confident predictions about the consequences of Brexit?
答案往往是否定的。戈夫提出的观点是，这些人从哪里开始对英国脱欧的后果做出如此自信的预测？
And the best empirical evidence would suggest that probably not materially more accurate than simple extrapolation algorithms.
最好的经验证据表明，这可能并不比简单的外推算法更准确。
It was brought up for a political reason.
它是出于政治原因提出的。
He had a political point to make.
他有一个政治观点要表达。
And Dominic Cummings had a pleasant political point to make as well.
多米尼克·卡明斯也提出了令人愉快的政治观点。
I feel that he fears that parts of the civil service are hostile toward Brexit and want to undermine the Boris Johnson administration objectives.
我觉得他担心部分公务员对英国退欧怀有敌意，并希望破坏鲍里斯·约翰逊政府的目标。
I think this comes to something deeper now.
我认为现在这已经涉及到更深层次的问题了。
It’s not just about the UK.
这不仅仅是关于英国。
It’s about the intellectual fissure that exists between social science and conservatives, that most social scientists are liberal, and conservatives are wary of advice from social scientists.
这是关于社会科学和保守派之间存在的智力鸿沟，大多数社会科学家都是自由派，而保守派对社会科学家的建议持谨慎态度。
I think we may have partly paid some price for that in this epidemiological .
我认为，在这种流行病学方面，我们可能已经为此付出了一些代价。
.
。
.
。
to put it kindly, the slowness of the Trump administration response to COVID-19.
客气地说，是特朗普政府对 COVID-19 的反应缓慢。
Yes.
是的。
You brought up Britain.
你养育了英国。
If we look back at speeches in the British House of Commons, who is giving the most cognitively complex speeches?
如果我们回顾一下英国下议院的演讲，谁的演讲认知最复杂？
Bob Putnam collected those data that I reported in that study in 1984.
鲍勃·普特南 (Bob Putnam) 收集了我在 1984 年的研究中报告的数据。
Bob Putnam collected original data and he reported in a book, Beliefs of Politicians, published in the 1970s.
鲍勃·普特南 (Bob Putnam) 收集了原始数据，并在 20 世纪 70 年代出版的《政治家的信仰》一书中进行了报道。
I think those data were based on interviews with members of the British House of Commons.
我认为这些数据是基于对英国下议院议员的采访。
They were not speeches.
它们不是演讲。
They were interviews, confidential interviews that Bob Putnam got access to and put a lot of work into obtaining.
这些是鲍勃·普特南（Bob Putnam）获得的采访，是保密的采访，并为此付出了大量的努力。
He shared them with me, and I used them as grist for a small research program I was running in the 1980s on cognitive style and political ideology, trying to tease apart the rigidity of the right versus the ideologue hypotheses.
他与我分享了它们，我将它们作为我在 20 世纪 80 年代运行的一个关于认知风格和政治意识形态的小型研究项目的基础，试图区分右派与意识形态假设的僵化。
And who sounds the smartest from that period?
那个时期谁听起来最聪明？
In that period it was a mixture of moderate labor, moderate conservative.
那个时期是温和劳动和温和保守的混合体。
It was the centrists that did better.
中间派做得更好。
It was slightly left shifted.
它稍微向左移动。
Do you think we can draw any inferences from that about politics?
您认为我们可以从中得出关于政治的任何推论吗？
Should we have more faith in the people who sound smarter or not at all?
我们应该对那些听起来更聪明的人更有信心还是根本不聪明？
Well, I think that particular measure in integrative complexity does have some correlation with forecasting accuracy.
嗯，我认为综合复杂性的特定衡量标准确实与预测准确性有一定的相关性。
But I think you’re picking up something more than just forecasting accuracy.
但我认为你学到的不仅仅是预测准确性。
You’re picking up what I call value pluralism.
你正在接受我所说的价值多元化。
You’re picking up a tendency to endorse values that are often in conflict with each other.
你开始倾向于认可经常相互冲突的价值观。
So the more frequently that you, as a political thinker, confront cognitive dissonance between your values, your value orientations, the more pressured you are to engage in integratively complex synthetic thinking.
因此，作为一名政治思想家，你越频繁地面对你的价值观和价值取向之间的认知失调，你就越有压力进行综合性复杂的综合思维。
When your values are more lopsided, it’s easier to engage in what we call simpler modes of cognitive dissonance reduction, like denial or bolstering and spreading of the alternatives.
当你的价值观更加不平衡时，你就更容易采取我们所说的更简单的减少认知失调的模式，比如否认或支持和传播替代方案。
So downplay one value, push up the other value, and make your life stress-free.
因此，淡化一种价值观，提高另一种价值观，让你的生活没有压力。
But some ideological positions at some points in history require more tolerance for dissonance and some people are more inclined to fill those roles.
但在历史的某些时刻，某些意识形态立场需要对不和谐有更多的容忍度，而有些人更倾向于填补这些角色。
And you think those politicians are also likely to be better forecasters?
您认为那些政客也可能是更好的预测者吗？
We’re not talking about huge effect sizes here or integrative complexity.
我们这里讨论的不是巨大的效应规模或综合复杂性。
I think that fluid intelligence is probably a more powerful predictor.
我认为流体智力可能是一个更强大的预测因子。
A combination of fluid intelligence and integrative complexity, I think, does boost up forecasting accuracy, yes.
我认为，流体智能和综合复杂性的结合确实提高了预测的准确性，是的。
And if you were running the CIA, those people who are pluralistic in the manner you just outlined, would you promote them more rapidly?
如果你正在管理中央情报局，那些以你刚才概述的方式多元化的人，你会更快地提拔他们吗？
Now of course, the CIA, bear in mind is, by statute, supposed to be value-neutral.
当然，请记住，根据法规，中央情报局应该是价值中立的。
They’re supposed to be feeding impartial, apolitical advice.
他们应该提供公正、非政治性的建议。
Just the —
只是——
But no one believes that, right?
但没有人相信这一点，对吗？
Well, that is supposed to be the division of labor here.
嗯，这应该就是这里的分工。
I think one reason they may be interested in forecasting tournaments is because forecasting tournaments incentivize people to do one thing and only one thing, and that’s accuracy.
我认为他们对预测比赛感兴趣的原因之一是预测比赛会激励人们做一件事，而且只做一件事，那就是准确性。
You don’t get points for skewing your judgements toward your favorite cause.
如果你的判断偏向于你最喜欢的事业，你不会得到分数。
You take a hit in the long term by doing that.
这样做从长远来看你会受到打击。
On reforming the CIA
关于改革中央情报局
If you were in charge of the CIA and had a free hand, how would you reform it?
如果你是中央情报局的负责人并且可以自由支配，你会如何改革它？
Whoa, boy!
哇，男孩！
Well, there’s a long history to efforts to reform the CIA.
嗯，改革中央情报局的努力有着悠久的历史。
It was founded in 1947, and there have been various efforts since then.
它成立于1947年，此后进行了各种努力。
People have been unhappy with the CIA for many reasons over time, Vietnam being a big one.
长期以来，人们对中央情报局的不满有很多原因，其中越南是其中一个重要原因。
But there are lots of other reasons that people have expressed unhappiness.
但人们表达不满的原因还有很多。
Both liberals and conservatives have, at various points, been unhappy with the performance of the intelligence community.
自由派和保守派都曾多次对情报界的表现感到不满。
In 2001, it came to a crisis point, I think, and then there was a commission to reform intelligence analysis.
我认为，2001 年，它到达了一个危机点，然后成立了一个改革情报分析的委员会。
After the WMD fiasco in Iraq, the pressure grew even more.
伊拉克大规模杀伤性武器惨败后，压力变得更大。
One reason why we’re even talking right now today, is that the intelligence community was forced, essentially by the recommendations of the reform commission, to take keeping score more seriously, to take training for accuracy and monitoring of accuracy more seriously.
我们今天谈论的原因之一是，情报界被迫（主要是改革委员会的建议）更加认真地对待记分、更加认真地对待准确性培训和准确性监测。
That’s when they created the Intelligence Advanced Research Projects Activity, which is the R&D branch housed within the Office of the Director of National Intelligence.
就在那时，他们创建了情报高级研究项目活动，这是国家情报总监办公室内的研发部门。
Its job is to support innovative research that will improve the quality of intelligence analysis.
其工作是支持创新研究，提高情报分析的质量。
That can be defined in various ways, but accuracy is certainly one very important component of that.
这可以通过多种方式来定义，但准确性无疑是其中一个非常重要的组成部分。
But it’s not accuracy with a liberal skew or a conservative skew.
但这并不是自由主义倾向或保守主义倾向的准确性。
It’s supposed to be just plain, “just the facts, ma’am” accuracy.
它应该很简单，“只是事实，女士”的准确性。
To predict.
预测。
That reflects values, obviously.
显然，这反映了价值观。
Which questions they choose to predict — that reflects values.
他们选择预测哪些问题——反映了价值观。
Whenever a bureaucracy tells me they’re value-free, I start getting more suspicious very quickly, right?
每当官僚机构告诉我他们没有价值时，我很快就会开始变得更加怀疑，对吗？
I never believe them.
我从来不相信他们。
It might be okay for them not to be value-free, but the cynical response is the correct one here.
对他们来说，不追求价值可能没有问题，但愤世嫉俗的反应在这里是正确的。
Well indeed.
确实如此。
The old expression, “There’s no view from nowhere.” There is no such thing as pure value neutrality.
古话说“无处可看”。不存在纯粹的价值中立这样的东西。
That doesn’t mean it’s not something worth aspiring to.
这并不意味着它不值得追求。
But you’re right, the values .
但你是对的，价值观。
.
。
.
。
Even if you had a perfectly objective forecasting tournament system, if you had people that are generating the questions promoting a political agenda, you could skew the results.
即使你有一个完全客观的预测锦标赛系统，如果你有人提出促进政治议程的问题，你也可能会扭曲结果。
I think that’s one of your points, right?
我认为这是你的观点之一，对吗？
On things under- and overrated
关于被低估和高估的事情
Now in the middle of all of these discourses, we have a segment called overrated versus underrated.
现在，在所有这些讨论的中间，我们有一个部分称为高估与低估。
I’ll toss out a few names, ideas, and you tell me if you think they’re overrated or underrated.
我会抛出一些名字、想法，然后你告诉我你是否认为它们被高估或低估了。
How’s that?
怎么样？
Okay.
好的。
Philadelphia, the city of Philadelphia — overrated or underrated?
费城，费城 ——被高估还是低估？
I got endless grief when my wife and I decided to leave Berkeley and move to Philadelphia.
当我和妻子决定离开伯克利搬到费城时，我感到无尽的悲伤。
People thought that we were borderline insane.
人们认为我们已经疯了。
But we left for very personal reasons, and without going into what those were, I would say Philadelphia has been a moderately pleasant surprise.
但我们离开是出于非常个人的原因，在不讨论这些原因的情况下，我想说费城是一个相当令人愉快的惊喜。
It’s a city that has many, many problems.
这是一个有很多很多问题的城市。
But it’s not as bad as the people in Northern California thought it was.
但情况并不像北加州人民想象的那么糟糕。
Tolstoy.
托尔斯泰。
I read a little bit of Tolstoy, and I’ve seen a number of films, and I know some of the shorthand versions, and I know that Isaiah Berlin had a hell of a time classifying Tolstoy as a hedgehog or a fox.
我读过一点托尔斯泰的作品，也看过很多电影，我知道一些速记版本，我知道以赛亚·伯林（Isaiah Berlin）将托尔斯泰归类为刺猬或狐狸时经历了一段痛苦的时光。
But I think I’ll pass on that.
但我想我会继续这一点。
John Cleese.
约翰·克里斯.
He’s commented on you.
他对你发表了评论。
You’re allowed to comment on him.
你可以对他发表评论。
I had a wonderful time as a kid watching Monty Python and also Fawlty Towers.
我小时候看《巨蟒剧团》和《福尔蒂塔》度过了一段美好的时光。
I’ve enjoyed his comedy.
我很喜欢他的喜剧。
I haven’t followed him since then, but when I was younger, I thought he was absolutely hilarious and brilliant.
从那以后我就没有关注他了，但当我年轻的时候，我认为他绝对是搞笑和才华横溢的。
And I appreciate the flattering things he said about my work.
我很欣赏他对我的工作所说的赞美之词。
The television show, The Sopranos.
电视节目《黑道家族》。
I fell for it.
我爱上了它。
James Gandolfini — I fell in love with his performances.
詹姆斯·甘多菲尼 — 我爱上了他的表演。
One of the last things he did is he played Leon Panetta in Zero Dark Thirty.
他做的最后一件事是在《零黑暗三十》中扮演莱昂·帕内塔。
And there he is, eliciting forecasts from people in the CIA about whether Osama bin Laden is in that compound in Abbottabad.
他就在那里，引起中央情报局人员的预测，猜测奥萨马·本·拉登是否就在阿伯塔巴德的那个大院里。
Is he there or isn’t he effing there?
他在场还是不在场？
Great, wonderful.
太棒了，太棒了。
I never knew him, but I think very highly of his work.
我从来不认识他，但我对他的工作评价很高。
The threat of terrorism?
恐怖主义的威胁？
Do we overrate it or underrate it in the United States?
在美国，我们是高估了它还是低估了它？
That’s a very difficult question because of the tail risk aspect to it.
这是一个非常困难的问题，因为它存在尾部风险。
If you look at the number of people who’ve died from terrorism versus other causes, it would seem that the amount of money we spend on suppressing terrorism would be disproportionate.
如果你看看死于恐怖主义的人数与其他原因相比，我们在镇压恐怖主义上花费的资金似乎是不成比例的。
But the tail risk complicates that a lot.
但尾部风险使情况变得更加复杂。
What is your favorite movie?
你最喜欢的电影是什么？
I don’t have a hierarchy like that, I’m sorry.
抱歉，我没有这样的等级制度。
Nothing comes to mind?
什么也没想到吗？
What’s the movie you’ve seen the greatest number of times, that you can count?
你看过最多次数（你能数出来）的电影是哪部？
I don’t know if I watch movies all that often over and over.
我不知道我是否经常一遍又一遍地看电影。
I did see myself coming back — just very recently, last week in the quarantine period — to Westworld.
我确实看到自己回来了——“就在最近，隔离期间的上周”——回到了《西部世界》。
It’s not a movie.
这不是电影。
It’s a series, of course.
当然，这是一个系列。
I think the first two seasons are quite brilliant.
我认为前两季非常精彩。
On historical counterfactuals, by what year do you think the ascent of the West was more or less inevitable?
关于历史反事实，您认为到哪一年西方的崛起或多或少是不可避免的？
Well, I have inside information here.
嗯，我这里有内幕消息。
We did a survey of some very prominent historians.
我们对一些非常著名的历史学家进行了调查。
We reported it in that book, Unmaking the West.
我们在那本书《毁灭西方》中对此进行了报道。
We put a part of it anyway, also in an article in American Political Science Review.
不管怎样，我们还是把其中的一部分放在了《美国政治科学评论》的一篇文章中。
If you looked at just the unweighted average of judgments from the median, I think was probably around 1730, 1740.
如果你只看中位数的未加权平均判断，我认为可能是在 1730 年、1740 年左右。
And you think after that, it was not very contingent?
你认为在那之后，这不是很偶然吗？
I’m not a historian of the West.
我不是西方历史学家。
What do I really know about that?
我对此到底了解多少？
I’m simply reporting the news here.
我只是在这里报道新闻。
And is there any great hinge of contingency that you think about looking backwards?
回顾过去，您是否考虑过任何重大的偶然事件？
Like, “Oh my goodness, if there hadn’t been a Reformation.” Or “If there hadn’t been a Council of Trent.” Or what?
就像，“天哪，如果没有宗教改革就好了。”或者“如果没有特伦特会议就好了。”或者什么？
Well, I’m a fan of Steve Pinker.
嗯，我是史蒂夫·平克的粉丝。
I think the Enlightenment was a big deal.
我认为启蒙运动是一件大事。
Because it got people thinking more rationally.
因为它让人们更加理性地思考。
Yeah.
是的。
More in terms of science.
更多的是在科学方面。
And that had huge spillover effects.
这产生了巨大的溢出效应。
Now, could there have been an enlightenment in China?
那么，中国能有启蒙吗？
Could the Chinese have created certain types of technologies without science?
中国人可以在没有科学的情况下创造出某些类型的技术吗？
What if they hadn’t scuttled their navy in around 1400?
如果他们没有在 1400 年左右凿沉他们的海军怎么办？
There are all those sorts of counterfactuals.
存在所有这些类型的反事实。
And how necessary or contingent do you think it is that we keep on thinking in Enlightenment-like terms?
您认为我们继续用类似启蒙运动的方式思考有多大必要性或偶然性？
Is it once you’re locked into it, it keeps on going?
一旦你被锁定，它就会继续下去吗？
Or is it like good government that you have to renew it every generation or two?
或者就像一个好的政府一样，你必须每隔一两代就更新一次？
Well, I see the work I’m doing as very much in the spirit of the Enlightenment.
嗯，我认为我正在做的工作非常符合启蒙运动的精神。
Public transparent standards of evidence for judging subject matter expertise.
用于判断主题专业知识的公开透明的证据标准。
I think one of the great challenges of our time is striking the right balance between democracy and technocracy.
我认为我们这个时代的巨大挑战之一是在民主与技术统治之间取得适当的平衡。
And I think the fissures that have emerged .
我认为裂痕已经出现。
.
。
.
。
It’s not just conservatives are skeptical of social science — and it’s currently in some parts of biological science too, which I think’s very unfortunate — but you see some reflexive skepticism towards science on the left as well.
不仅保守派对社会科学持怀疑态度——而且目前在生物科学的某些部分也持怀疑态度，我认为这是非常不幸的——而且你也会看到左派对科学有一些本能的怀疑。
So, how do you manage the relation between small-d democrats and technocrats in a society in which expert guidance is increasingly crucial?
那么，在专家指导日益重要的社会中，如何处理小民主派和技术官僚之间的关系呢？
What are you learning from playing the game Civilization V, or at least watching others do so?
您从玩《文明 V》游戏或至少观看其他人这样做时学到了什么？
That’s a good example actually, of why I’m not a superforecaster.
这实际上是一个很好的例子，说明了为什么我不是超级预测家。
I don’t play Civilization V.
我不玩文明V。
But Civilization V was one of the simulations that IARPA chose to feature in its counterfactual forecasting tournaments under the rubric of FOCUS.
但《文明 V》是 IARPA 选择在其 FOCUS 标题下的反事实预测锦标赛中展示的模拟之一。
And if any of your listeners are interested in signing up to be forecasters for FOCUS.
如果您的任何听众有兴趣注册成为 FOCUS 的预测员。
We still have one more round, round 5, and we will be recruiting people.
我们还有一轮，第五轮，我们将招募人员。
But again, you see, it reflects my temperament.
但你看，这又反映了我的气质。
I don’t have the patience for a game like Civ V.
我对《文明V》这样的游戏没有耐心。
Forecasting is inevitably a mixture of fluid and crystallized intelligence, and you have to invest a lot of energy into mastering a game like Civilization V.
预测不可避免地是流动情报和结晶情报的混合体，你必须投入大量精力来掌握像《文明 V》这样的游戏。
And I suppose, as people get older, they may become less likely to make those kinds of cognitive investments.
我想，随着人们年龄的增长，他们可能会变得不太可能进行此类认知投资。
It becomes more and more essential as I get older, I think, to focus on the things where I have a real comparative advantage.
我认为，随着年龄的增长，专注于我真正具有比较优势的事情变得越来越重要。
The best chess players are all young, right?
最好的棋手都是年轻人，对吗？
This we know.
这我们知道。
There’s clear data.
有明确的数据。
Well, yeah.
嗯，是的。
It’s interesting the domains in which child prodigies emerge: music and chess and math.
有趣的是，神童出现的领域是：音乐、国际象棋和数学。
If we take supercounterfactualists and superforecasters, do those two groups basically overlap?
如果我们以超级反事实主义者和超级预测者为例，这两个群体基本上是重叠的吗？
Or how do they differ?
或者说它们有何不同？
I think they have to be rather intimately connected, although disentangling this one is going to be really, really hard.
我认为它们必须相当密切地联系在一起，尽管要解开这一点将非常非常困难。
It’s one of the things I do want to dedicate a few years of my life to doing.
这是我确实想花几年时间去做的事情之一。
Now obviously, when you say someone’s a good counterfactualiser, some people shrug their shoulders, and they say, “Well, how are you possibly going to know whether you would’ve gotten to undo the assassination of the archduke in 1914?
显然，当你说某人是一个很好的反事实者时，有些人会耸耸肩，然后说：“好吧，你怎么可能知道你是否能够撤销 1914 年刺杀大公的事件呢？
Do you undo World War I, undo Hitler?
你会撤销第一次世界大战、撤销希特勒吗？
You undo World War II.
你撤销了第二次世界大战。
You make Kennedy grouchier during the Cuban Missile Crisis, trigger World War III.”
你让肯尼迪在古巴导弹危机期间更加不满，引发了第三次世界大战。”
You’ve got these sorts of arguments that are essentially unresolvable.
这些争论本质上是无法解决的。
You can’t rerun history.
你无法重演历史。
You can rerun Civilization V.
您可以重新运行《文明 V》。
But you can’t rerun history.
但你无法重演历史。
So that makes counterfactuals a place where ideologues can retreat.
因此，这使得反事实成为理论家可以撤退的地方。
They can make up the data, make up whatever facts they want to justify pretty much whatever.
他们可以编造数据，编造任何他们想要证明的事实。
No matter how bad the war in Iraq went, you can always argue that things would have been worse if Saddam Hussein had remained in power.
无论伊拉克战争有多糟糕，你总是可以说，如果萨达姆·侯赛因继续掌权，情况会更糟。
So you have these factual and counterfactual reference points that people use in debates implicitly to make rhetorical points.
因此，人们在辩论中隐含地使用这些事实和反事实的参考点来提出修辞观点。
Part of what attracted to me to counterfactuals was (A) how important they are in drawing any lessons from history, (B) how important they are in policy arguments, and © how unresolvable they are.
反事实吸引我的部分原因是（A）它们对于从历史中汲取教训有多么重要，（B）它们在政策争论中有多重要，以及©它们是多么难以解决。
One of the things we’re hoping to do in the FOCUS program is to develop some objective metrics for identifying people and methods of generating probabilities that produce superior counterfactual forecasts in simulated worlds in which you can rerun history and assess what the probability distributions of possible wars are.
我们希望在 FOCUS 项目中做的一件事是开发一些客观指标来识别人员和生成概率的方法，从而在模拟世界中产生出色的反事实预测，在模拟世界中您可以重新运行历史并评估可能发生战争的概率分布。
Well, it turns out, you get World War I 37 percent of the time even if you undo the assassination of the archduke.
好吧，事实证明，即使你撤销了对大公的刺杀，你也有 37% 的机会会爆发第一次世界大战。
And you get something like World War II .
你会遇到类似第二次世界大战的事情。
.
。
.
。
You see where we’re going.
你知道我们要去哪里。
So we’re hoping that one result of FOCUS will be to help us identify people and methods that generate superior counterfactual forecasts in domains where there is a ground truth.
因此，我们希望 FOCUS 的结果之一是帮助我们识别在有基本事实的领域中产生卓越的反事实预测的人员和方法。
The next task will be to connect superior performance in simulated worlds to superior performance in the actual world.
下一个任务是将模拟世界中的卓越性能与现实世界中的卓越性能联系起来。
And this is where things get tricky, of course, because in the actual world, we don’t have the ground truth.
当然，这就是事情变得棘手的地方，因为在现实世界中，我们没有基本事实。
So if I ask you a counterfactual question of the form, if NATO hadn’t expanded eastward as far as it did in 2004 under the Baltics, NATO-Russia relations would be considerably friendlier than they are now.
因此，如果我问你一个反事实的问题，如果北约没有像2004年在波罗的海国家那样向东扩张，北约与俄罗斯的关系将会比现在友好得多。
Are chess players good forecasters?
国际象棋棋手是优秀的预测者吗？
But do you want to go with this for a second?
但你想先讨论一下这个吗？
No, I don’t want to stop on that one.
不，我不想就此停止。
Yeah, keep on going.
是的，继续吧。
Now, we can’t rerun history.
现在，我们无法重演历史。
We don’t know how relations with Russia would be if NATO hadn’t gone into the Baltics.
我们不知道如果北约没有进入波罗的海地区，与俄罗斯的关系将会如何。
“Russia would have gobbled up the Baltics.” “Maybe Russia would be friendly or feel less threatened.” You have people who have more hawkish or more dovish mental models of Russia, and those mental models predispose them to give you certain canned, almost ideologically reflective answers to those counterfactuals, right?
“俄罗斯会吞并波罗的海国家。” “也许俄罗斯会表现得友好，或者感到更少的威胁。”有些人对俄罗斯有更鹰派或更鸽派的思维模式，而这些思维模式使他们倾向于对这些反事实给出某些现成的、几乎在意识形态上反思的答案，对吗？
Right.
正确的。
But you can measure what people’s beliefs are in the counterfactual.
但你可以通过反事实来衡量人们的信念。
And then you can measure people’s beliefs about conditional forecasts that are logically connected to the counterfactuals in a kind of a Bayesean entrance network.
然后你可以衡量人们对条件预测的信念，这些预测在逻辑上与贝叶斯入口网络中的反事实相关。
If I know that you think that the Russians would be every bit as nasty and snarly, even if we hadn’t moved into the Baltic — they might have even been nastier — it’s probably a fair bet that you’re also likely to think it’s a good idea to increase arm sales to the Ukraine, ratchet up sanctions on Putin’s cronies, and so forth.
如果我知道你认为俄罗斯人会同样肮脏和恶毒，即使我们没有进入波罗的海——“他们甚至可能更肮脏”——那么你很可能会认为增加对乌克兰的军售、加大对普京亲信的制裁等等是个好主意。
So we can identify the counterfactual belief correlates to more or less accurate conditional forecasting.
因此，我们可以识别与或多或少准确的条件预测相关的反事实信念。
And in that sense, you can indirectly validate or invalidate, you can render more or less plausible certain counterfactual beliefs.
从这个意义上说，你可以间接验证或无效，你可以使某些反事实信念或多或少合理。
That’s the longer-term objective of this research program.
这是该研究计划的长期目标。
We’re not playing Civilization V for the sake of getting better at Civilization V.
我们玩《文明 5》并不是为了更好地玩《文明 5》。
The ultimate goal is to link the sophistication of counterfactual reasoning about the past to the subtlety in the accuracy of conditional forecast going into the future.
最终目标是将关于过去的反事实推理的复杂性与对未来的条件预测的准确性的微妙性联系起来。
Another thing you should observe by the way — if people are becoming better counterfactual reasoners, you should observe less ideological polarization in their counterfactual beliefs.
顺便说一句，你应该观察到的另一件事是——如果人们正在成为更好的反事实推理者，你应该观察到他们的反事实信念中的意识形态极化更少。
The counterfactual belief should become as ideologically depolarized as conditional forecasts are.
反事实信念应该像有条件的预测一样在意识形态上去极化。
Does playing World of Warcraft a lot help you become a better forecaster, or playing chess?
经常玩魔兽世界或下国际象棋是否可以帮助您成为更好的预测者？
I don’t have any evidence bearing on either of those things.
我没有任何与这两件事有关的证据。
There’s a third variable problem there, too.
这里还存在第三个变量问题。
But you get used to a test, right?
但你已经习惯了测试，对吧？
You know if you lose?
输了你知道吗？
There’s very little self-deception.
很少有自欺欺人的情况。
This is true.
这是真实的。
The people who do well in these sorts of things often like games like that.
擅长这类事情的人往往喜欢这样的游戏。
Venture capitalists — when they try to spot talent in others, do you interpret their behavior in terms of a superforecaster model?
风险投资家 — 当他们试图发现他人的才能时，你是否用超级预测模型来解释他们的行为？
Someone like Peter Thiel — he found Mark Zuckerberg, Reed Hoffman, Elon Musk.
像彼得·蒂尔这样的人——他找到了马克·扎克伯格、里德·霍夫曼、埃隆·马斯克。
He’s a kind of superforecaster.
他是一位超级预测家。
How do you superforecast talent in other people?
你如何超级预测其他人的才能？
Well, it really helps to be working in an environment in which super bright people are not super rare.
嗯，在一个超级聪明的人并不罕见的环境中工作确实很有帮助。
It’s very, very hard to forecast, to identify talent when the base rate falls below 1 in 1,000, 1 in 10,000, 1 in 100,000.
当基本比率低于千分之一、万分之一、十万分之一时，就很难预测和识别人才。
You’re looking for a needle in a haystack.
你正在大海捞针。
But the great advantage that venture capitalists in Silicon Valley have is that the talent pool is relatively rich.
但硅谷风险投资家的一大优势就是人才储备比较丰富。
There’s quite a few flaky people that come by seeking their money, for sure.
当然，有不少古怪的人来寻求他们的钱。
But their odds of success are significantly better than they would be if they’re working from a population base rate.
但他们的成功几率明显高于按照人口基数工作的几率。
And of course, they can tolerate a lot of mistakes because just a few hits will pay for a lot of false positives.
当然，他们可以容忍很多错误，因为只需几次点击就会付出很多误报的代价。
But some do much, much better than others, right?
但有些人做得比其他人好得多，对吗？
Mike Moritz, Peter Thiel.
迈克·莫里茨，彼得·泰尔。
Right, the question is, are they doing better because they have better social networks?
是的，问题是，他们做得更好是因为他们拥有更好的社交网络吗？
Or they are doing better, they have better judgment?
或者他们做得更好，他们有更好的判断力？
Superforecasting as a technique for predicting the future — does it also apply to predicting how successful people will be?
超级预测作为一种预测未来的技术 ——它也适用于预测人们的成功程度吗？
Yes.
是的。
I think everything we know, from the overlap between superforecasting and intelligence and the overlap between intelligence and success in many different professional lines of work, would suggests that that’s almost certainly true.
我认为我们所知道的一切，从超级预测和情报之间的重叠，以及许多不同专业工作中情报和成功之间的重叠，都表明这几乎肯定是正确的。
Who first superforecasted your own major success?
谁第一个超级预测了你自己的重大成功？
I don’t know what my first major success was.
我不知道我的第一个重大成功是什么。
Some people might challenge that there has been a major success in my career.
有些人可能会质疑我的职业生涯取得了重大成功。
Who first saw it coming?
谁第一个看到它的到来？
I think probably my advisor — University of British Columbia — Peter Suedfeld who supported me and believed in me when there didn’t seem to be very many good reasons for doing so.
我想可能是我的导师 — 不列颠哥伦比亚大学 — 彼得·苏德菲尔德（Peter Suedfeld）在似乎没有太多充分理由这样做的情况下支持我并相信我。
I was a confused Canadian kid who was unsure whether to become a lawyer in Canada or go off to Oxford or go to graduate school in the US, who tipped me towards social science and US.
我是一个困惑的加拿大孩子，不确定是否要在加拿大成为一名律师，还是去牛津大学或去美国读研究生，他让我倾向于社会科学和美国。
And what did he see that other people had not seen?
他看到了什么其他人没有看到的？
God only knows.
只有上帝知道。
Well, if anyone would know it’s you, right?
好吧，如果有人知道那就是你，对吧？
You’ve lived with it for some time.
你已经忍受它有一段时间了。
I think he thought I was probably bright enough to do well, and he probably thought that I was contrarian and weird enough that there was some possibility of doing something distinctively well.
我认为他认为我可能足够聪明，可以做得很好，而且他可能认为我是逆向思维和奇怪的人，有可能把一些事情做得特别好。
Something distinctive and different and doing it well.
一些独特和不同的东西并且做得很好。
Do you think academic advisors, in general, today undervalue weird students?
您认为现在的学术顾问是否普遍低估了奇怪的学生？
There’s probably a tendency in that direction, yes.
是的，可能有朝这个方向发展的趋势。
Because weirdness is a big word.
因为奇怪是一个很大的词。
Weirdness takes lots of forms that you and I .
你和我的怪异有很多种形式。
.
。
.
。
We would not be embarrassed about turning away lots of weird people.
我们不会因为拒绝很多奇怪的人而感到尴尬。
Sure.
当然。
Do you think people who grow up with a second culture are better forecasters?
您认为在第二种文化中长大的人是更好的预测者吗？
Oh, you’re thinking of my work with Carmit Tadmor.
哦，你在想我和卡米特·塔德莫的工作。
Of course.
当然。
Gosh, you’ve really looked into my vita here, right?
天哪，你真的看过我的简历，对吧？
This is highly unusual.
这是非常不寻常的。
I think there does seem to be some advantage there, yeah.
我认为这似乎确实有一些优势，是的。
And where does that come from?
这是从哪里来的？
How does it work?
它是如何运作的？
It ties into accountability.
它与责任相关。
Accountability to conflicting audiences and value pluralism.
对相互冲突的受众负责并重视多元化。
You have a richer internal dialogue.
你有更丰富的内部对话。
You learn to balance conflicting perspectives more.
您学会更多地平衡相互冲突的观点。
You have to be a better perspective taker.
你必须成为一个更好的观点接受者。
Perspective taking’s a very important part of superforecasting, too.
换位思考也是超级预测的一个非常重要的部分。
How do we create more Nate Silvers and Philip Tetlocks?
我们如何创造更多的 Nate Silvers 和 Philip Tetlocks？
What should we change in the world to get more of you?
为了得到更多的你，我们应该改变世界什么？
Well, I think Nate and I are probably very different creatures.
嗯，我认为内特和我可能是非常不同的生物。
Sure, but we want more of you both, right?
当然可以，但是我们想要你们俩更多，对吧？
What should we do?
我们应该做什么？
One thing that would be useful .
一件有用的事情。
.
。
.
。
There is this tendency for training in universities to become hyperprofessionalized and compartmentalized.
大学的培训存在过度专业化和条块化的趋势。
So it is harder for people who have weird interests that straddle, say, psychology and organizational science and political science and law and history.
因此，对于那些对心理学、组织科学、政治科学、法律和历史等领域有奇怪兴趣的人来说，这就更困难了。
The people who have weird sets of interests — it’s hard for them to get traction in the current career environment.
那些有着奇怪兴趣的人——他们很难在当前的职业环境中获得吸引力。
Certainly, in my home discipline of psychology, it had this kind of rampant publication inflation in which PhD students would be very lucky to get a job.
当然，在我的家乡心理学学科中，存在着这种猖獗的出版物膨胀，在这种情况下，博士生能找到工作就很幸运了。
You’d have to have a ridiculous number of publications and some of them in Ajournals.
你必须拥有大量的出版物，其中一些是在 Ajournals 上。
It’s the sort of thing we didn’t really expect 30 years ago.
这是 30 年前我们并没有真正预料到的事情。
We thought, “Oh, that’s a tenure case.
我们想，“哦，这是一个终身职位的案例。
Oh, that’s a junior hire.” Those kinds of pressures I think produce a narrowing of focus.
哦，那是初级员工。”我认为这些压力会导致关注范围缩小。
Now you say, “Well, yes .
现在你说：“嗯，是的。
.
。
.” I said earlier a lot of the great advances in science come from hedgehogs.
”。我之前说过，科学上的许多伟大进步都来自刺猬。
We’re producing hedgehogs on an industrial scale here.
我们正在这里以工业规模生产刺猬。
And I think there’s some advantage that may make a little bit more room for weirdo eclectics.
我认为有一些优势可以为怪异不拘一格的人提供更多空间。
And the way that departments are carved up — would you change that at all?
部门划分的方式 ——你会改变吗？
A number of academics in the past, like Jim March in UC Irvine many decades ago, tried to do something like that.
过去的许多学者，比如几十年前加州大学欧文分校的吉姆·马奇（Jim March），都尝试过做类似的事情。
Amy Gutmann, the president of the University of Pennsylvania, is trying to do that with Penn Integrates Knowledge, which is the chaired professorships that were used, among other things, to hire Barb and me and hire a number of other people.
宾夕法尼亚大学校长艾米·古特曼 (Amy Gutmann) 正试图通过宾夕法尼亚大学整合知识来实现这一目标，该教授职位除其他外，还被用来雇佣巴布和我，以及雇佣其他一些人。
And we are sort of floaters.
我们是漂浮者。
We’re not connected to one unit.
我们没有连接到一台设备。
We have multiple connections.
我们有多个联系。
That’s a very hospitable work environment from my point of view.
从我的角度来看，这是一个非常友善的工作环境。
But I don’t see a lot of places doing a Jim March–UC Irvine experiment.
但我没有看到很多地方在进行吉姆·马奇-加州大学欧文分校的实验。
“Well, let’s integrate all social sciences together.” And Amy Gutmann, PIK, Penn Integrates Knowledge kind of program — you don’t see too many of them yet.
“好吧，让我们把所有社会科学整合在一起。”还有 Amy Gutmann、PIK、Penn Integrates Knowledge 之类的项目 — 你还没有看到太多这样的项目。
It runs against the grain.
它与谷物背道而驰。
I’m not even sure it’ll survive at Penn beyond Amy.
我什至不确定它是否会在艾米之后在宾夕法尼亚大学生存。
The natural tendency will be for departments to want to claw back the resources.
部门自然会想要收回资源。
Let’s say someone comes up to you and they say, “Philip, I would like to be more fox-like.
假设有人走过来对你说：“菲利普，我想变得更像狐狸。
I’m not enough of a fox.” What actual advice would you give them to achieve that?
我还不够像一只狐狸。”为了实现这一目标，您会给他们什么实际建议？
What should they do?
他们应该做什么？
Or not do?
还是不做？
Wake up earlier in the morning?
早上起得早吗？
Exercise more?
多运动？
I had to commit career suicide to become more fox-like.
为了变得更像狐狸，我不得不自杀。
Maybe they’re not an academic.
也许他们不是学者。
They’re a smart business person.
他们是聪明的商人。
How do they do this?
他们是如何做到这一点的？
Kind of obvious things, like read a little bit more outside your field.
一些显而易见的事情，比如在你的领域之外多读一点书。
If you’re a liberal, read the Wall Street Journal.
如果您是自由主义者，请阅读《华尔街日报》。
If you’re a conservative, read the New York Times.
如果您是保守派，请阅读《纽约时报》。
Expose yourself to dissonant points of view.
让自己接触不和谐的观点。
Try to cultivate some interests outside your field, and try to connect them together.
尝试培养一些你领域之外的兴趣，并尝试将它们联系在一起。
I think there’s an optimal distance.
我认为有一个最佳距离。
For history, for example — sounds quite different from what I did.
例如，对于历史来说，听起来与我所做的完全不同。
I started as an experimental psychologist, and history looks very different, but they can be connected because historical judgment is something that psychologists study to some degree.
我最初是一名实验心理学家，历史看起来非常不同，但它们可以联系起来，因为历史判断是心理学家在某种程度上研究的东西。
Psychologists are interested in hindsight and counterfactuals and so forth.
心理学家对事后诸葛亮和反事实等等感兴趣。
So you can link the two.
所以你可以将两者联系起来。
I think there’s an optimal distance, and when you go foraging as a fox, you probably don’t want to forage way, way far away.
我认为存在一个最佳距离，当你像狐狸一样去觅食时，你可能不想在很远的地方觅食。
You want to forage far enough away that it’ll be stimulating but still possible to reconnect.
你想要在足够远的地方寻找食物，这样会很刺激，但仍然有可能重新建立联系。
What kinds of people are best at adversarial collaboration?
什么样的人最擅长对抗性合作？
Those are rare people also.
这样的人也很稀有。
It’s really hard to do.
这确实很难做到。
But how do you spot them?
但你如何发现它们呢？
Is it a personality trait?
这是一种人格特质吗？
A cognitive trait?
认知特征？
Gosh, that’s a hard one.
天哪，这太难了。
Danny Kahneman, I think, coined the term when he was dealing with his various critics over time.
我认为，丹尼·卡尼曼（Danny Kahneman）在与各种批评者打交道时创造了这个词。
My wife, Barb, actually was involved in an adversarial collaboration between the Kahneman camp and the Gigerenzer camp on the conjunction fallacy.
我的妻子巴布实际上参与了卡尼曼阵营和吉格伦泽阵营之间关于合取谬误的对抗性合作。
It took at least two or three years of her life.
这至少花了她两三年的时间。
It’s very hard to get people to .
让人们参与是非常困难的。
.
。
.
。
Well, lots of us, in principle, are Popperians.
嗯，原则上我们很多人都是波普尔主义者。
We believe in stating our beliefs as falsifiable hypotheses.
我们相信将我们的信念表述为可证伪的假设。
Most of us believe our beliefs are probabilistic, so we’re somewhat Bayesian.
我们大多数人都相信我们的信念是概率性的，所以我们有点贝叶斯主义。
But that’s lip service.
但这只是口头上的服务。
There’s what we believe.
这就是我们所相信的。
There’s an epistemological formal self-concept, which is kind of noble, falsificationist, and probabilistic.
有一种认识论形式的自我概念，它是一种崇高的、证伪主义的和概率性的。
And then there’s how we actually behave when our egos are at stake in particular controversies.
然后是当我们的自我在特定争议中受到威胁时，我们的实际行为如何。
And those things are quite different.
这些事情是完全不同的。
I think Danny Kahneman who’s not known as an optimist, proposed it.
我认为丹尼·卡尼曼（Danny Kahneman）提出了这个建议，他并不是一个乐观主义者。
It sounds like an optimistic idea, but I think he’s not all that optimistic about what it can achieve on close inspection.
这听起来像是一个乐观的想法，但我认为他对仔细检查后所能取得的成果并不那么乐观。
My efforts at adversarial collaboration have not been all that successful.
我在对抗性合作方面的努力并没有那么成功。
I’d like to jumpstart a few of them again, but it’s very hard to find the right dance partners.
我想再次跳起其中的几个，但很难找到合适的舞伴。
Let’s try a question from the realm of the everyday and the mundane.
让我们尝试一个来自日常生活和世俗领域的问题。
If I go around, and I look at Mexican restaurants, I’m very good at predicting which ones have excellent tacos.
如果我四处走走，看看墨西哥餐馆，我很擅长预测哪些餐馆的炸玉米饼很棒。
What are you good at predicting?
你擅长预测什么？
I’m not good at predicting your questions.
我不擅长预测你的问题。
What am I good at predicting?
我擅长预测什么？
I think I was pretty good at anticipating the fragility of a lot of microsocial science knowledge prior to the replication crisis erupting.
我认为我很擅长在复制危机爆发之前预见到许多微观社会科学知识的脆弱性。
No, I mean everyday life.
不，我指的是日常生活。
Oh, okay.
哦好的。
So not in my —
所以不在我的——
Not social events.
不是社交活动。
Not social science.
不是社会科学。
What in your life are you good at predicting?
在你的生活中你最擅长预测什么？
When you’re going to get tired and wanting to go to bed at night?
晚上什么时候你会感到疲倦并想睡觉？
Or when the dog wants to eat?
或者当狗想吃东西的时候？
What is it?
它是什么？
Well, we have a pet-free existence.
好吧，我们的生活没有宠物。
Our lives are actually pretty simple.
我们的生活其实很简单。
It gets me subscribed to that old adage: You’ll be boring and staid in your life so you can be violent and creative in your work.
这让我认同了那句古老的格言：你的生活会变得无聊和古板，所以你可以在工作中变得暴力和富有创造力。
We have kind of a routine here.
我们这里有一种惯例。
That was highly predictable.
这是高度可预测的。
And now, in the quarantine days .
现在，在隔离期间。
.
。
.
。
It used to be, pre-quarantine, “Oh, we’re going to go to Europe.
过去，在隔离前，“哦，我们要去欧洲。
We’re going to go here or there.” There were the little points of unpredictability that spiced up life, but those do not exist right now.
我们要去这里或那里。”有些不可预测的小事给生活增添了乐趣，但现在这些都不存在了。
And two more questions to close.
还有两个问题要结束。
First, what can you tell us about your next project?
首先，您能告诉我们您的下一个项目吗？
Well, I think the next project is the one that I mentioned earlier.
嗯，我认为下一个项目就是我之前提到的那个。
It’s linking historical counterfactual reasoning with conditional forecasting.
它将历史反事实推理与条件预测联系起来。
I think it’ll be the second phase of the FOCUS research tournaments.
我认为这将是 FOCUS 研究锦标赛的第二阶段。
I think that counterfactual reasoning has, for too long, been the last refuge of ideological scoundrels.
我认为反事实推理长期以来一直是意识形态恶棍的最后避难所。
Insofar as we can improve the standards of evidence of proof in judging counterfactual claims as well as conditional forecasts and linking the two, I think there’s a potential for improving the quality of debates among interested parties.
只要我们能够提高判断反事实主张以及条件预测并将两者联系起来的证据标准，我认为就有可能提高有关各方之间辩论的质量。
And finally, what should a superforecaster predict about the future course of your influence?
最后，超级预测者应该如何预测你的影响力的未来走向？
Oh, to be very cautious because we know we’re running against the grain.
哦，要非常谨慎，因为我们知道我们正在逆势而行。
We’re running against a psychological grain.
我们正在对抗心理谷物。
We’re running against human nature.
我们正在违背人性。
We’re running against a sociological grain.
我们正在对抗社会学的规律。
But see, enlightenment is stable you tell us, right?
但是你看，你告诉我们启蒙是稳定的，对吧？
If it keeps on cumulating and growing, your influence should be enormous given your other presuppositions.
如果它不断积累和增长，考虑到你的其他预设，你的影响力应该是巨大的。
Well, there’s a lot of cognitive resistance to treating one’s beliefs as falsifiable, probabilistic propositions.
好吧，将一个人的信念视为可证伪的概率命题存在很多认知阻力。
People naturally gravitate toward thinking of their beliefs as ego-defining, quasi-sacred possessions.
人们自然而然地倾向于将自己的信仰视为自我定义的、准神圣的财产。
That’s one major obstacle.
这是一个主要障碍。
Then you have the existing status hierarchies.
然后您就拥有了现有的状态层次结构。
You have subject matter experts who are entrenched, have influence.
你们拥有根深蒂固、具有影响力的主题专家。
Why would they want to participate in exercises in which the best possible outcome is a tie?
为什么他们想要参加最好的结果是平局的练习？
They reaffirm that they deserve the status that they already have.
他们重申他们应得的地位。
So psychological, sociological resistance.
所以心理上、社会上的抵抗。
It’s interesting, sociologists and economists have different reactions to forecasting tournaments.
有趣的是，社会学家和经济学家对预测比赛有不同的反应。
Sociologists — they react, “Why would anyone be naive enough to think that anyone would want to have a forecasting tournament in their organization?” They’re status disruptive, right?
社会学家 ——他们的反应是：“为什么会有人天真地认为有人会想在他们的组织中举办预测锦标赛呢？”他们破坏了地位，对吗？
Yes.
是的。
And an economist would say, “Well, if these things are so great, how come they’re not everywhere?”
经济学家会说：“好吧，如果这些东西如此伟大，为什么它们不是无处不在呢？”
And with that, Philip Tetlock, thank you very much.
菲利普·泰洛克，非常感谢你。
It’s been a pleasure.
这是一种乐趣。
Take care.
小心。