Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft)

Itamar Gilad: You fake it, you do a fake door test, you do a smoke test, Wizard of Oz tests.

Itamar Gilad：你假装它，你做了假门测试，你做了烟雾测试，绿野仙踪测试。

We used a lot of those in the tabbed inbox by the way, one of the first early versions was actually we showed the tabbed inbox working to people.

顺便说一句，我们在选项卡式收件箱中使用了很多这些功能，第一个早期版本实际上是我们向人们展示了选项卡式收件箱的工作原理。

But it wasn't really Gmail, it was just a facade of HTML and behind the scenes and according to the permissions that the users gave us some of us moved just the subject and the sender into the right place.

但它并不是真正的 Gmail，它只是 HTML 的一个外观，在幕后，根据用户授予我们的权限，我们中的一些人只是将主题和发件人移到了正确的位置。

So initially the interviewer kind of distracted them and then showed them their inbox and then the top 50 messages were sorted to the right place more or less if we got it right.

因此，最初面试官会分散他们的注意力，然后向他们展示收件箱，然后如果我们做对了，前 50 条消息或多或少会被分类到正确的位置。

And people were like, "Wow, this is actually very cool." But it gave us some evidence to go and say, "Hey, we should try and build this thing."

人们会说，“哇，这实际上非常酷。”但它给了我们一些证据，让我们可以说：“嘿，我们应该尝试建造这个东西。”

Lenny: Welcome to Lenny's Podcast where I interview world-class product leaders and growth experts to learn from their hard won experiences, building and growing today's most successful products.

Lenny：欢迎来到 Lenny 的播客，我在这里采访了世界一流的产品领导者和增长专家，学习他们来之不易的经验，构建和发展当今最成功的产品。

Today my guest is Itamar Gilad.

今天我的客人是伊塔玛·吉拉德。

Itamar, is a product coach, author, speaker and former longtime product manager at Google where you worked on Gmail, identity and YouTube.

Itamar 是一名产品教练、作家、演讲家，也是 Google 的前长期产品经理，您曾在 Google 从事 Gmail、身份识别和 YouTube 方面的工作。

He also just published an awesome new book called Evidence-Guided: Creating High-Impact Products in the Face of Uncertainty.

他还刚刚出版了一本很棒的新书，名为《证据引导：面对不确定性创造高影响力产品》。

Itamar, has an important perspective on why and also how you can push your team and organization from an opinion-based decision-making process to a more evidence guided approach.

Itamar 对于为什么以及如何推动您的团队和组织从基于意见的决策过程转向更多证据引导的方法有着重要的观点。

In our conversation, Itamar, shares a number of very practical and handy frameworks that do just that including the confidence meter, metrics trees, GIST and the GIST board, plus his take on how people often misuse ICE for prioritizing ideas.

在我们的谈话中，Itamar 分享了许多非常实用且方便的框架，这些框架正是这样做的，包括置信度表、指标树、GIST 和 GIST 板，以及他对人们如何经常滥用 ICE 来确定想法优先顺序的看法。

Also, how you could make your OKRs more effective and so much more.

此外，如何使 OKR 更有效等等。

Enjoy this episode with Itamar Gilad, after a short word from our sponsors.

在我们的赞助商简短致辞后，请与 Itamar Gilad 一起欣赏这一集。

This episode is brought to you by Ezra, the leading full body cancer screening company.

本集由领先的全身癌症筛查公司 Ezra 为您带来。

Lenny: I actually used Ezra, earlier this year unrelated to this podcast completely on my own dime because my wife did one and loved it and I was super curious to see if there's anything that I should be paying attention to in my body as I get older.

Lenny：我实际上用过 Ezra，今年早些时候与这个播客完全无关，完全是我自己出钱，因为我妻子做了一个并且很喜欢它，我非常好奇随着年龄的增长，我的身体是否有什么东西应该注意。

The way it works is you book an appointment, you come in, you put on some very cool silky pajamas that they give you that you get to keep afterwards.

它的运作方式是你预约，进来，穿上他们给你的一些非常酷的丝质睡衣，之后你可以保留这些睡衣。

You go into an MRI machine for 30 to 45 minutes and then about a week later you get this detailed report sharing what they found in your body.

您进入 MRI 机器 30 到 45 分钟，大约一周后，您会收到一份详细报告，分享他们在您体内发现的情况。

Luckily, I had what they called an unremarkable screening which means they didn't find anything cancerous.

幸运的是，我接受了他们所谓的“无异常筛查”，这意味着他们没有发现任何癌变的东西。

But they did find some issues in my back which I'm getting checked out at a physical next month probably because I spend so much time sitting in front of a computer.

但他们确实发现了我背部的一些问题，我将在下个月的体检中接受检查，可能是因为我花了太多时间坐在电脑前。

Half of all men will have cancer at some point in their lives, as will one third of women.

一半的男性会在一生中的某个时刻患上癌症，三分之一的女性也会患癌症。

Half of all of them will detect it late.

其中一半人会发现得很晚。

Lenny: According to the American Cancer Society, early cancer detection has an 80% survival rate compared to less than 20% for late stage cancer.

Lenny：根据美国癌症协会的数据，早期癌症检测的存活率高达 80%，而晚期癌症的存活率不到 20%。

The Ezra, team has helped 13% of their customers identify potential cancer early and 50% of them identify other clinically significant issues such as aneurysms, disc herniations, which may be is what I have or fatty liver disease.

Ezra 团队帮助 13% 的客户及早发现了潜在的癌症，其中 50% 的客户发现了其他具有临床意义的问题，例如动脉瘤、椎间盘突出（这可能就是我所患的疾病）或脂肪肝。

Ezra, scans for cancer and 500 other conditions in 13 organs using a full body MRI powered by AI and just launched the world's only 30-minute full body scan which is also their most affordable.

Ezra 使用人工智能驱动的全身 MRI 扫描 13 个器官的癌症和​​ 500 种其他病症，并刚刚推出了世界上唯一的 30 分钟全身扫描，这也是他们最实惠的。

Their scans are non-invasive and radiation free and Ezra, is offering listeners $150 off their first scan with code Lenny150.

他们的扫描是非侵入性且无辐射的，Ezra 为听众提供 150 美元的首次扫描优惠，使用代码 Lenny150。

Book your scan at ezra.com/lenny.

请访问 ezra.com/lenny 预订扫描。

That's E-Z-R-A.com/lenny.

那是 E-Z-R-A.com/lenny。

This episode is brought to you by Vanta, helping you streamline your security compliance to accelerate your growth.

本集由 Vanta 为您带来，帮助您简化安全合规性以加速您的成长。

Thousands of fast-growing companies like Gusto, Quorum, Quora and Modern Treasury, trust Vanta to help build, scale, manage and demonstrate their security and compliance programs and get ready for audits in weeks not months.

Gusto、Quorum、Quora 和 Modern Treasury 等数千家快速发展的公司信任 Vanta 帮助构建、扩展、管理和演示其安全与合规计划，并在几周而不是几个月内为审计做好准备。

By offering the most in-demand security and privacy frameworks such as SOC 2, ISO 27001, GDPR, HIPAA, and many more.

通过提供最受欢迎的安全和隐私框架，例如 SOC 2、ISO 27001、GDPR、HIPAA 等。

Lenny: Vanta, helps companies obtain the reports they need to accelerate growth, build efficient compliance processes, mitigate risks to their businesses and build trust with external stakeholders.

Lenny：Vanta 帮助公司获得加速增长、建立高效合规流程、降低业务风险以及与外部利益相关者建立信任所需的报告。

Over 5,000 fast-growing companies use Vanta to automate up to 90% of the work involved with SOC 2 and these other frameworks.

超过 5,000 家快速发展的公司使用 Vanta 自动化 SOC 2 和其他框架所涉及的高达 90% 的工作。

For a limited time Lenny's Podcast listeners get $1,000 off Vanta.

Lenny's Podcast 听众限时享受 Vanta 1,000 美元优惠。

Go to vanta.com/lenny, that's V-A-N-T-A.com/lenny to learn more and to claim your discounts get started today.

请访问vanta.com/lenny（即 V-A-N-T-A.com/lenny）了解更多信息并立即开始领取折扣。

Itamar, thank you so much for being here.

伊塔玛，非常感谢你来到这里。

Welcome to the podcast.

欢迎来到播客。

Itamar Gilad: It's a pleasure being here, thank you for inviting me.

Itamar Gilad：很高兴来到这里，感谢您邀请我。

Lenny: It's my pleasure.

莱尼：这是我的荣幸。

I thought we'd start with the story of your work on Google+ and Gmail and how those experiences formed your perspective on how to build a successful product.

我想我们应该从您在 Google+ 和 Gmail 上的工作故事开始，以及这些经历如何形成您对如何构建成功产品的看法。

Can you share that story?

你能分享那个故事吗？

Itamar Gilad: Google+ was my first experience at Gmail, I joined Gmail in August 2011 and the first thing they asked me is, "Let's connect Gmail with Google+." If you're hazy about the story, back then Facebook was massive.

Itamar Gilad：Google+ 是我第一次使用 Gmail，我于 2011 年 8 月加入 Gmail，他们问我的第一件事是，“让我们将 Gmail 与 Google+ 连接起来。”如果你对这个故事不太了解，那么当时的 Facebook 规模非常庞大。

It's still massive but then it was growing like mushrooms, people were spending hours.

它仍然很大，但后来它像蘑菇一样生长，人们花了几个小时。

That really freaked out Google and the obvious solution was to launch a social network of Google called Google+ and we all believe in this thing, it really caught on very well initially we all used it, we all believed in it.

这真的吓坏了谷歌，显而易见的解决方案是推出一个名为 Google+ 的谷歌社交网络，我们都相信这个东西，它真的很流行，最初我们都使用它，我们都相信它。

So our mission was to build this thing and Google really cut no costs.

所以我们的使命是打造这个东西，而谷歌确实没有削减任何成本。

It created a whole new division within Google and it created a whole strategy around Google+ and we had to connect Gmail and YouTube and search to Google+ to make them more personalized in a sense and more social.

它在 Google 内部创建了一个全新的部门，并围绕 Google+ 创建了一个整体战略，我们必须将 Gmail 和 YouTube 以及搜索连接到 Google+，以使它们在某种意义上更加个性化、更具社交性。

So that was the idea and we went on and we launched a series of features in Gmail for a couple of years, honestly and Google+ itself became this massive project, very feature rich and with a lot of redesigns and iterations and none of it worked.

这就是我们的想法，我们持续了几年，在 Gmail 中推出了一系列功能，老实说，Google+ 本身成为了一个庞大的项目，功能非常丰富，进行了大量的重新设计和迭代，但没有一个起作用。

Itamar Gilad: It turned out people actually didn't need another social network, people didn't love it, people didn't use it.

Itamar Gilad：事实证明，人们实际上不需要另一个社交网络，人们不喜欢它，人们不使用它。

Eventually in Gmail we rolled back all the Google+ integration a few years later and Google+ itself was shut down in 2019.

最终，几年后，我们在 Gmail 中回滚了所有 Google+ 集成，Google+ 本身也于 2019 年关闭。

So putting aside all the tremendous waste that went into this, all the millions of person hours and personal weeks.

因此，抛开这其中的所有巨大浪费，所有数以百万计的工时和个人周。

In hindsight, not only did Google bet on the wrong thing it missed much easier opportunities.

事后看来，谷歌不仅押错了赌注，而且还错失了更容易的机会。

So just not far from Google's headquarters there was WhatsApp, not very famous in the US but they actually created massive impact.

离谷歌总部不远的地方就有WhatsApp，它在美国不是很有名，但实际上产生了巨大的影响。

Hundreds of millions of people were using their stuff and they became a threat to Facebook much more than Google was.

数亿人正在使用他们的产品，他们对 Facebook 的威胁比谷歌还要大得多。

So Google missed the opportunity of social mobile apps like WhatsApp, like Snapchat, etc and for me this story kind of was the epitome of what I call today, opinion-based development.

因此，谷歌错过了 WhatsApp、Snapchat 等社交移动应用程序的机会，对我来说，这个故事就是我今天所说的基于意见的开发的缩影。

We come up with an idea, we believe in it, all the indications show it's good.

我们提出一个想法，我们相信它，所有迹象都表明它是好的。

Itamar Gilad: Maybe the early tests show it's good, then we just go all in and we try to implement it and I made this very mistake many times as the product manager, I was the guy pushing for the ideas.

Itamar Gilad：也许早期的测试表明它很好，然后我们就全力以赴，尝试实现它，作为产品经理，我多次犯了这个错误，我是推动这些想法的人。

So for me, this was kind of a turning point I felt we need to adopt a different system.

所以对我来说，这是一个转折点，我觉得我们需要采用不同的系统。

Lenny: And just before you move on to the next story, how big was the team?

莱尼：在你开始下一个故事之前，团队有多大？

Roughly how many years was spent on this area?

这个领域大概花了多少年的时间？

Just to give people a sense of the waste as you said.

只是为了让人们感受到你所说的浪费。

Itamar Gilad: So there was a tremendous earthquake inside Google to create the Google+ team, teams and the entire divisions were kind of thrown apart and reformatted and I think at its peak it was about 1000 people inside-

Itamar Gilad：因此，Google 内部发生了一场巨大的地震，创建了 Google+ 团队，团队和整个部门都被拆散并重新格式化，我认为在高峰期，内部大约有 1000 人——

Itamar Gilad: It was a division the size of Android and Docs and a really sizable thing, they're under their own buildings.

Itamar Gilad：这个部门的规模相当于 Android 和 Docs，而且规模非常大，他们在自己的大楼下。

It's taken from the playbook of Steve Jobs, create this whole secretive project inside and just run like hell.

它取自史蒂夫·乔布斯的剧本，在内部创建整个秘密项目，然后拼命运行。

Lenny: Yeah.

莱尼：是的。

I remember though Facebook was really scared, I remember they shut everything down.

我记得虽然 Facebook 真的很害怕，但我记得他们关闭了一切。

It as like a code DEFCON one situation too, so it really scared Facebook at the same time.

它就像代码 DEFCON 一样，也是一种情况，所以它同时也吓坏了 Facebook。

Itamar Gilad: Yeah, it's true.

伊塔玛·吉拉德：是的，确实如此。

But at the end of the day, neither Google's advertising revenue was affected, neither was Facebook affected.

但最终，谷歌的广告收入没有受到影响，Facebook 也没有受到影响。

So it turned out this idea was not that necessary after all.

结果发现这个想法根本就没有必要。

Lenny: Yeah, okay.

莱尼：是的，好的。

So that's an example of something that didn't work because it was opinion based software, I think the phrase you used and then there's a different experience with tabs I think with Gmail.

这是一个不起作用的例子，因为它是基于意见的软件，我认为您使用了这个短语，然后我认为 Gmail 的标签体验有所不同。

Itamar Gilad: That's right.

伊塔玛·吉拉德：没错。

So Google, is a very successful company.

所以谷歌是一家非常成功的公司。

It's not for me to criticize it or to in hindsight kind of say you guys need to be better and some of the people that were behind Google+ was some of the smartest leaders and I still think they are despite this story.

我不应该批评它，也不应该事后说你们需要变得更好，Google+ 背后的一些人是最聪明的领导者，尽管有这个故事，我仍然认为他们是。

If you look back at the history of Google, how things started in the first decade or so.

如果你回顾一下谷歌的历史，你会发现它在最初十年左右的时间里是如何开始的。

Google, was what I call an evidence guided company.

谷歌，就是我所说的一家以证据为导向的公司。

So essentially it put a high premium on focusing on customers, coming up with a lot of ideas on looking at the data, looking at how these ideas actually worked out.

因此，从本质上讲，它高度重视以客户为中心，通过研究数据提出很多想法，看看这些想法的实际效果如何。

They weren't shy about launching betas and things that were very rough and incomplete and learning from that and then they expected people to take action based on the results.

他们并不羞于推出测试版以及非常粗糙和不完整的东西，并从中学习，然后他们期望人们根据结果采取行动。

So fail fast is a very famous paradigm and so you had to kill your project or pivot it seriously if it didn't work out and I think had we kept fail fast it would've really have helped Google+, if we had this mentality.

因此，快速失败是一个非常著名的范例，因此，如果项目没有成功，你就必须终止项目或认真调整它，我认为，如果我们保持快速失败，如果我们有这种心态，它确实会对 Google+ 有所帮助。

Itamar Gilad: But for some reason with Google+, Google put this playbook aside and used a different playbook which I call plan and execute essentially.

Itamar Gilad：但由于 Google+ 的某种原因，Google 将这个剧本放在一边，并使用了另一个剧本，我称之为计划和执行。

But I think inside Google the DNA still existed.

但我认为在谷歌内部，DNA 仍然存在。

So inside Gmail, the next project after Google+ was the tabbed inbox.

因此，在 Gmail 中，继 Google+ 之后的下一个项目是选项卡式收件箱。

So it was kind of the reverse of Google+, it started as a very small idea that no one believed in and we started looking what's behind the city?

所以这有点与 Google+ 相反，它一开始只是一个非常小的想法，没有人相信，然后我们开始寻找城市背后的东西？

What's the goal?

目标是什么？

What's the problem actually we're trying to solve?

我们真正想要解决的问题是什么？

It turned out that a lot of people were receiving social notifications and promotions, etc, and most of them were very passive.

事实证明，很多人都在收到社交通知、促销活动等，而且大多数人都非常被动。

They weren't clearing their inbox, they were just living in this world of clutter and I came up with an idea how to fix this.

他们没有清理收件箱，他们只是生活在这个混乱的世界中，我想出了一个解决这个问题的想法。

I was sure it was great, I wanted to push it, plan and execute, but my colleagues were like, "Hold on, we actually tried this.

我确信这很棒，我想推动它、计划和执行，但我的同事们说，“等等，我们实际上尝试过这个。

We have a bunch of ideas to help people organize their inbox, they're not using it.

我们有很多想法可以帮助人们组织收件箱，但他们没有使用它。

Why is your idea good?"

为什么你的主意好？”

Itamar Gilad: So that sent us, kind of me and my team into researching these users into establishing a goal that was much more user-centric and then thinking of other ideas.

Itamar Gilad：因此，这促使我们、我和我的团队研究这些用户，建立一个更加以用户为中心的目标，然后考虑其他想法。

And then we started testing them much more rigorously and basically we started testing on our own inboxes and then we recruited other dog footers, other Googlers to test the same inbox, then we put it outside for external testers.

然后我们开始更严格地测试它们，基本上我们开始在我们自己的收件箱上进行测试，然后我们招募其他狗脚手、其他谷歌员工来测试同一个收件箱，然后我们将其放在外面供外部测试人员使用。

We did usability studies, we did data, we built a whole data mining team and a whole machine learning team to build the right categorization and we ended up with a solution that turned out to be very successful for a lot of these passive users.

我们做了可用性研究，我们做了数据，我们建立了一个完整的数据挖掘团队和一个完整的机器学习团队来构建正确的分类，我们最终得到了一个对于许多被动用户来说非常成功的解决方案。

This was a surprise to a lot of people because most of my colleagues and most of the people I talk with actually know how to manage their inbox.

这让很多人感到惊讶，因为我的大多数同事和与我交谈的大多数人实际上都知道如何管理他们的收件箱。

So for them that solution makes complete nonsense, like splitting promotions and social to the side sounds like the stupidest idea.

因此，对他们来说，这个解决方案完全是无稽之谈，比如将促销和社交分开听起来是最愚蠢的想法。

But there's about 85% of the population, 85 to 88% that absolutely love it and today Gmail has about 1.8 billion active users according to Gmail.

但大约 85% 的人（85% 到 88%）绝对喜欢它，根据 Gmail 的数据，如今 Gmail 拥有大约 18 亿活跃用户。

Itamar Gilad: Most of these users are using this feature, so it was a pretty high impact feature as well.

Itamar Gilad：大多数用户都在使用此功能，因此它也是一个影响很大的功能。

Lenny: And the feature specifically, just in case people aren't totally getting it is the promotions folder and the social I think and then the regular.

莱尼：具体的功能是，以防万一人们没有完全理解它，我认为是促销文件夹和社交文件夹，然后是常规文件夹。

Itamar Gilad: Yeah, there are a couple more that you can enable in settings if you like.

Itamar Gilad：是的，如果您愿意，您还可以在设置中启用更多功能。

Lenny: Yeah, I use it, I love it.

莱尼：是的，我用它，我喜欢它。

Except it puts my newsletter in people's promotions folder, who do I talk to about that?

除了将我的时事通讯放入人们的促销文件夹中之外，我该与谁谈论这件事？

Itamar Gilad: Yeah.

伊塔玛·吉拉德：是的。

Newsletters are a very complicated scenario for the categorization engine.

对于分类引擎来说，时事通讯是一个非常复杂的场景。

Lenny: Yeah.

莱尼：是的。

We just need an exception for my newsletter and then we're good.

我们只需要为我的时事通讯提供一个例外，然后就可以了。

Okay, but go on.

好吧，但继续吧。

Itamar Gilad: So in hindsight I was asking and saying, "Why was this project so different?" And I think the reason is that we didn't have that much confidence in our opinions.

Itamar Gilad：所以事后看来，我问并说：“为什么这个项目如此不同？”我认为原因是我们对自己的观点没有那么大的信心。

We had opinions, we had ideas but we didn't just go all in and just let's build it.

我们有意见，有想法，但我们并没有全力以赴，只是让我们来构建它。

We actually used an evidence guided system and I think that's not unique just to Google.

我们实际上使用了证据引导系统，我认为这不仅仅是谷歌独有的。

I think every successful product company out there that you look at Amazon, Airbnb, anyone you will check, at least in their best periods they found a way to balance human judgment with evidence.

我认为，亚马逊、Airbnb 等每家成功的产品公司，至少在它们最好的时期，都找到了一种平衡人类判断与证据的方法。

They didn't try to obliterate human judgment and opinion just to supercharge them with evidence and they came up with very different models.

他们并没有试图消除人类的判断和意见，只是为了给他们提供证据，他们提出了非常不同的模型。

Apple, is another example but the principle still holds in all of these companies.

苹果是另一个例子，但这一原则仍然适用于所有这些公司。

Lenny: Awesome.

莱尼：太棒了。

So you took that experience and all the experience you've had from coaching product leaders working with companies and you wrote this book called Evidence-Guided, which people on YouTube could see sitting there behind you.

因此，您汲取了这段经验以及您在指导与公司合作的产品负责人时获得的所有经验，写了这本书，名为《证据引导》，YouTube 上的人们可以在您身后看到这本书。

So I want to talk through some of these stories and then some of these other lessons and frameworks that emerged.

所以我想谈谈其中的一些故事，然后谈谈出现的一些其他教训和框架。

But maybe just to start, what's the elevator pitch for this book?

但也许只是为了开始，这本书的电梯推销是什么？

Itamar Gilad: So this is a book for people like us, product people who want to bring evidence guided thinking or modern product management if you like into their organizations.

Itamar Gilad：所以这本书适合像我们这样的产品人员，如果您愿意的话，他们希望将证据引导的思维或现代产品管理带入他们的组织。

There's a lot of challenges, it's not simple, we all read the books, we all know the theory, we all know some parts of the system.

有很多挑战，这并不简单，我们都读过书，我们都知道理论，我们都知道系统的某些部分。

It tries to give you a system how to do that, it's a meta framework that kind of helps you lift your organization in the direction of evidence guidance if that's what you want to do.

它试图为您提供一个如何做到这一点的系统，它是一个元框架，可以帮助您将您的组织提升到证据指导的方向，如果这就是您想要做的。

Lenny: So going back to the story briefly before we get into the frameworks and lessons of the book.

莱尼：所以在我们进入这本书的框架和教训之前，先简单回顾一下这个故事。

In the first example of Google+, basically it came top down, "Hey, we need to build a social network, go build it." Obviously that happens at a lot of companies, I don't know if there's an easy answer to this.

在 Google+ 的第一个例子中，基本上是自上而下的，“嘿，我们需要建立一个社交网络，去建立它吧。”显然，很多公司都会发生这种情况，我不知道是否有一个简单的答案。

But are there cases where it does make sense to approach it that way?

但在某些情况下，以这种方式处理确实有意义吗？

Obviously Apple is a classic example of Steve Jobs, is like we need to build an iPhone.

显然苹果是史蒂夫·乔布斯的典型例子，就像我们需要打造一部iPhone一样。

I don't know if that's exactly how it went.

我不知道事情到底是不是这样。

But are there instances where it is worth just approaching new product ideas that way based on the experience and creativity and insights of the founder?

但在某些情况下，是否值得根据创始人的经验、创造力和洞察力来实现新产品创意？

Or is your thinking it should always come from this evidence-based approach?

或者您认为它应该始终来自这种基于证据的方法？

Itamar Gilad: I think the founders are very important, especially in the startup and scale-ups phase.

Itamar Gilad：我认为创始人非常重要，特别是在初创和规模化阶段。

They come up with many of the most important ideas and it's super important that they have the space to express and to push the organization to look at those.

他们提出了许多最重要的想法，他们有空间表达并推动组织关注这些想法，这一点非常重要。

However, it's not about shutting them down it's about looking at them critically.

然而，这并不是要关闭它们，而是要批判性地看待它们。

You need to create the environment in the organization where the leader comes and says, "You know what?

你需要在组织中创造一个环境，让领导者过来并说：“你知道吗？

I talked to these three customers, I figured it out.

我和这三位顾客交谈过，我明白了。

Here's what we need to do in the next five years." And you need to ask, "Where's your evidence?" And by the way, the example you give that's a classic example.

这就是我们未来五年需要做的事情。”你需要问：“你的证据在哪里？”顺便说一句，你举的例子是一个经典的例子。

Steve Jobs, he just brainstorm in his kitchen the iPhone and then just told the team to build it.

史蒂夫·乔布斯（Steve Jobs），他只是在厨房里集思广益，设计了 iPhone，然后告诉团队去建造它。

That's the story Steve Jobs, told but it's not the real story at all.

这是史蒂夫·乔布斯讲述的故事，但这根本不是真实的故事。

Now we know what actually happened and the iPhone has actually a story of discovery, of trial and error, multiple projects to do it, multitouch with phones, most of them failed.

现在我们知道到底发生了什么，iPhone 实际上有一个发现、试验和错误的故事，多个项目来实现它，手机的多点触控，其中大多数都失败了。

Itamar Gilad: Steve Jobs, was the architect.

伊塔玛·吉拉德：史蒂夫·乔布斯，是建筑师。

He kind of managed to connect the dots and eventually come up with this perfect device but he wasn't actually the creator, it wasn't his brainchild.

他设法将这些点联系起来，并最终想出了这个完美的设备，但他实际上并不是创造者，这不是他的创意。

He was actually against it for a while but over time as he saw the evidence, as he saw what this thing can do, as he saw the demos he was able to piece together something that was very useful.

他实际上有一段时间反对它，但随着时间的推移，当他看到证据，当他看到这个东西可以做什么，当他看到演示时，他能够拼凑出一些非常有用的东西。

Lenny: That's really important insight.

莱尼：这是非常重要的见解。

People that are hearing this might feel like I like this idea of pushing back and encouraging the founders to make it more evidence guided.

听到这句话的人可能会觉得我喜欢这种抵制并鼓励创始人使其更有证据指导的想法。

In the case of say Google+, was it even possible?

就 Google+ 而言，这可能吗？

Could you have come to Larry and Sergey and be like, "Here's all this data I've gathered that tells us this is not going to work?" Do you have any advice for how to push back and encourage the founders and execs to really take the counterpoint seriously or really kind of vet their idea?

你是否可以对拉里和谢尔盖说：“这是我收集到的所有数据，告诉我们这行不通？”对于如何阻止和鼓励创始人和高管真正认真对待反对意见或真正审查他们的想法，您有什么建议吗？

Itamar Gilad: So another nice thing about Google is that it's a very open culture and people are not shy to tell even Sergey and Larry that they are wrong and they do this all the time.

Itamar Gilad：谷歌的另一个好处是，它是一种非常开放的文化，人们甚至会毫不羞涩地告诉谢尔盖和拉里他们错了，而且他们一直这样做。

In certain forms, right?

以某些形式，对吗？

You need to know the right channels.

您需要了解正确的渠道。

But there was a very big discussion about Google+ and whether it's the right thing to create a clone of Facebook, there was a very public internal discussion.

但是，关于 Google+ 以及创建 Facebook 克隆是否正确的问题，存在着非常大的讨论，还有非常公开的内部讨论。

I think what I would change is not have this discussion based on opinions, because when you have the discussion you come with your own opinions usually the most senior person's opinions will win.

我认为我要改变的是不要根据意见进行讨论，因为当你进行讨论时，你会提出自己的意见，通常最资深的人的意见会获胜。

That's just the way it is.

事情就是这样。

If we had come with hard data and we said, "Listen, things are not actually panning out the way you guys are expecting.

如果我们拿到了确凿的数据，然后我们说：“听着，事情实际上并没有按照你们预期的方式发展。

What can we do?

我们能做什么？

Should we continue?

我们应该继续吗？

Should we pivot this?" I think the discussion would've done better.

我们应该转向这个吗？”我认为讨论会做得更好。

Now I'm doing a huge disservice, I was not in all the discussions.

现在我造成了巨大的伤害，我没有参与所有的讨论。

I know probably in Google+, there were very serious discussions happening along these lines.

我知道在 Google+ 中可能发生过非常严肃的讨论。

Itamar Gilad: But it's just as a general trend, I find that evidence is very empowering for us smaller people in the organization or mid-level managers to be empowered to challenge the opinions.

Itamar Gilad：但这只是一个总体趋势，我发现证据非常有利于我们组织中的小人物或中层管理人员有权挑战观点。

Lenny: Is there anything tactically you found to be useful and effective in giving people, say they don't work at Google.

莱尼：您发现有什么战术上有用且有效的东西可以让人们说他们不在谷歌工作吗？

They work at companies where founders and bosses and execs are not as open to challenge.

他们所在的公司的创始人、老板和高管不太愿意接受挑战。

any tactically found about how to present a counter proposal or like, "Hey, I have this data that we should really pay attention to?"

任何关于如何提出反建议的战术性发现，或者诸如“嘿，我有这些我们真正应该关注的数据？”

Itamar Gilad: I think if you come with data, if you run a secret experiment and you come back and you show them you usually get one of two results.

Itamar Gilad：我认为，如果你带来数据，如果你进行秘密实验，然后回来向他们展示，你通常会得到两个结果之一。

Either they get extremely mad at you and they tell you to get back to work and to do what you were told and in that case, probably you need to start polishing your resume and look for another place either inside the organization or outside it because that person is not being reasonable to be honest.

要么他们对你非常生气，然后告诉你回去工作，按照吩咐去做，在这种情况下，你可能需要开始完善你的简历，并在组织内部或外部寻找另一个职位，因为说实话，那个人不讲道理。

But the more common case is they're pleasantly surprised and that's what happened with Steve Jobs, as well.

但更常见的情况是他们感到惊喜，史蒂夫·乔布斯也遇到过这种情况。

He was against phones but then people showed him all sorts of evidence that Apple can make a phone.

他反对手机，但后来人们向他展示了苹果可以制造手机的各种证据。

He was against multitouch initially but then he changed his mind, there was a lot of back and forth.

他最初反对多点触控，但后来他改变了主意，有很多来回。

So even, Steve Jobs, given evidence was willing to flip and I say this in many organizations.

因此，即使是史蒂夫·乔布斯，只要有证据，他也愿意翻转，我在许多组织中都这么说。

So evidence is so powerful, that's why this is the principle I based the book on.

所以证据是如此有力，这就是为什么这就是我写这本书的原则。

Lenny: You have this concept of being evidence guided.

莱尼：你有以证据为指导的概念。

People listening may feel like, "Hey, we're evidence guided, we're in experiments, we make decisions using data." Oftentimes they aren't actually and so what are signs that maybe you're not actually that evidence guided or as evidence guided as you think you are?

听的人可能会觉得，“嘿，我们有证据指导，我们正在进行实验，我们使用数据做出决策。”通常情况并非如此，那么什么迹象表明您实际上并没有像您认为的那样受到证据指导或证据指导？

Itamar Gilad: I think there's a few telltale signs that I look for, first the goals are very unclear.

Itamar Gilad：我认为我正在寻找一些明显的迹象，首先目标非常不清楚。

Either there are many or they're very kind of obscure and vague or they are about output, there's misalignment.

要么有很多，要么它们非常晦涩和模糊，要么它们与输出有关，存在偏差。

So the goals part is not there, usually this goes hand in hand with metrics.

所以目标部分不存在，通常这与指标齐头并进。

Missing metrics or just using revenue and business metric but there's no user facing metrics.

缺少指标或仅使用收入和业务指标，但没有面向用户的指标。

So that's another telltale sign, then there's a lot of time and effort spent on planning especially on road mapping.

这是另一个明显的迹象，需要花费大量时间和精力进行规划，尤其是路线图绘制。

Creating the perfect roadmap which really can consume a lot of time of the top management and PMs, etc. Then as you go down you see there's not a lot of experimentation and if there is experimentation there's not a lot of learning and finally another telltale sign is that the team is disengaged.

创建完美的路线图确实会消耗高层管理人员和产品经理等的大量时间。然后当你往下走时，你会发现没有太多的实验，即使有实验，也没有太多的学习，最后另一个明显的迹象是团队脱离了。

So the engineers are kind of getting the signal that what they need to do is deliver, they're focused on output, that's what they're measured on.

因此，工程师们得到了这样的信号：他们需要做的是交付，他们专注于输出，这就是他们的衡量标准。

So they're kind of disengaged, they're disengaged from the users, from the business, they don't care that much.

所以他们有点脱离，脱离用户、脱离业务，他们不太关心。

Itamar Gilad: It's usually something that you can fix by adopting a more evidence guided system.

Itamar Gilad：通常可以通过采用更多证据引导的系统来解决这个问题。

Lenny: Okay.

莱尼：好的。

So let's dive into your approach to becoming more evidence guided.

因此，让我们深入探讨一下如何变得更加有证据为指导。

In the book, you share this model that you call the GIST model which is kind of this overarching approach to building a product that almost forces you to be more evidence guided.

在书中，您分享了这个称为 GIST 模型的模型，这是构建产品的总体方法，几乎​​迫使您更多地以证据为指导。

So let's just start with what's the simplest way to understand this GIST model?

那么让我们首先了解理解这个 GIST 模型的最简单方法是什么？

Itamar Gilad: With your permission, I can show a few slides.

Itamar Gilad：如果您允许，我可以展示几张幻灯片。

Lenny: Here we go, and then yeah, a good excuse to go check this out on YouTube.

Lenny：我们开始吧，然后是的，这是一个很好的借口去 YouTube 上看看。

Itamar Gilad: All right, you're seeing this?

Itamar Gilad：好吧，你看到这个了吗？

So this is the GIST model, goals, ideas, steps and tasks, and essentially it's tries to break the change which is a really big change for a lot of companies into four slightly more manageable parts.

这就是 GIST 模型、目标、想法、步骤和任务，本质上它试图将变革分解为四个稍微更易于管理的部分，这对于很多公司来说是一个非常大的变革。

They're still big but each one you can tackle on its own and that's kind of the reason I kind of split it, and goals are about defining what we're trying to achieve, ideas are hypothetical ways to achieve the goals, steps are ways to implement the idea and validate it at the same time.

它们仍然很大，但每一个你都可以单独解决，这就是我将其分开的原因，目标是定义我们想要实现的目标，想法是实现目标的假设方法，步骤是实现想法并同时验证它的方法。

So essentially build, measure learn loops and tasks are the things we manage in Kanban and Jira and all these good tools.

因此，本质上，构建、衡量学习循环和任务是我们在看板和 Jira 以及所有这些优秀工具中管理的事情。

These are the things that your development team is usually very focused on and just listening to this, a lot of this will sound familiar to you because GIST is not a brand new invention.

这些是你的开发团队通常非常关注的事情，只要听听这些，你就会觉得其中很多听起来很熟悉，因为 GIST 并不是一个全新的发明。

It's a meta framework that puts in place a lot of existing methodologies.

它是一个元框架，采用了许多现有的方法。

It's based on lean startup, on design thinking, product discovery, growth, There's a lot of all of these things here.

它基于精益创业、设计思维、产品发现、成长，这里有很多这些东西。

It just tries to put them all into one framework or one model.

它只是试图将它们全部放入一个框架或一种模型中。

Lenny: So what's the simplest way to think about what this model is meant for?

Lenny：那么思考这个模型的用途的最简单方法是什么？

Is this how you think about your roadmap?

这就是您对路线图的看法吗？

Is this how you plan?

这就是你的计划吗？

What is this trying to tell people to do differently in the way they build product broadly?

这是想告诉人们以不同的方式广泛构建产品的方式是什么？

Itamar Gilad: I would say these are four areas that you need to look at and ask, are we doing the right thing in each?

伊塔马尔·吉拉德：我想说，你需要关注并询问这四个领域，我们在每个领域都在做正确的事情吗？

In each you may need to change or even transform and as I go and explain each one of those I'll give you basically three things.

在每一项中，你可能都需要改变甚至转变，当我解释每一项时，我将基本上给你三件事。

In each chapter in the book I try to touch on three things.

在本书的每一章中，我都尝试触及三件事。

The principles behind them, the frameworks or models that implement the principles and then process and the process honestly is the most brittle part and the one that you would need to change and adapt to your company.

它们背后的原则，实施这些原则然后进行流程的框架或模型，以及诚实的流程是最脆弱的部分，也是您需要更改和适应您的公司的部分。

Because not two companies are exactly the same, and it's very tempting when you write a book not to give any process but that's the part that people actually want the most.

因为没有两家公司是完全相同的，当你写一本书时不提供任何流程是很诱人的，但这是人们实际上最想要的部分。

So it's included as well, but just be aware that you will have to change this process.

因此它也包含在内，但请注意您必须更改此过程。

Lenny: Awesome.

莱尼：太棒了。

Okay, so we're going to talk about each of these four layers.

好的，我们将分别讨论这四层。

Before we do that, where do vision and strategy fit into this?

在我们这样做之前，愿景和战略在哪里适合这个？

Do they bucket into one of these four layers and how do you think about strategy and vision?

他们是否属于这四个层次之一？您如何看待战略和愿景？

Itamar Gilad: That's a great question, so there's this whole strategic context that is outside of GIST.

Itamar Gilad：这是一个很好的问题，所以整个战略背景都不在 GIST 之外。

GIST, is not trying to tackle that, it assumes it's in place, there's another huge blob which is research.

GIST 并没有试图解决这个问题，它假设它已经就位，还有另一个巨大的问题就是研究。

GIST, is not about research it's more about discovery and delivery.

GIST 不在于研究，而在于发现和交付。

But strategy is extremely important and you can use some of the tools we will talk about to develop your strategy as well.

但策略非常重要，您也可以使用我们将讨论的一些工具来制定您的策略。

In many companies the strategy is just a roadmap on steroids, it's small plan and execute just on a grand scale and Google+ again, was a strategic choice actually if you think about it.

在许多公司中，战略只是一个路线图，它是一个小计划，只是大规模执行，如果你仔细想想，Google+实际上也是一个战略选择。

So in the book there is a chapter where I touch on strategy and I explain how the same evidence guided methods are being used by companies to develop their strategy as well.

因此，书中有一章涉及战略，并解释了公司如何使用相同的证据引导方法来制定战略。

Lenny: Awesome, maybe one last context question.

莱尼：太棒了，也许是最后一个上下文问题。

So people might be seeing this and thinking okay cool, I have goals, I have ideas steps, I have tasks, I'm already doing this.

所以人们可能会看到这一点并认为好吧，酷，我有目标，我有想法步骤，我有任务，我已经在这样做了。

What is this kind of a counter or reaction to?

这是对什么的反击或反应？

What are people probably missing when they're seeing this and they're like, "Oh, I see.

当人们看到这一点时可能会错过什么，他们会说：“哦，我明白了。

This is like what we're not doing and this is the most important, this is something we should probably change." And we'll go through these in detail too.

这就像我们没有做的事情，这是最重要的，这是我们应该改变的事情。”我们也将详细介绍这些内容。

Itamar Gilad: I think talking about each one will help.

Itamar Gilad：我认为谈论每一个都会有所帮助。

Itamar Gilad: But we can talk about in each level what's actually being done.

Itamar Gilad：但是我们可以讨论每个级别实际正在做什么。

So when people say I have goals, usually they take the goals layer and use it as a planning session.

因此，当人们说我有目标时，通常他们会采用目标层并将其用作计划会议。

They talk about what shall we build by when, what are the resources?

他们讨论我们什么时候应该建设什么，有什么资源？

And that's actually not goals at all, that's planning work.

实际上这根本不是目标，而是计划工作。

Lenny: Cool, let's talk about goals and I know part of this is OKRs related too, so I'm excited to hear your take on OKRs.

Lenny：很好，我们来谈谈目标，我知道其中一部分也与 OKR 相关，所以我很高兴听到您对 OKR 的看法。

Itamar Gilad: Oh, that's a whole different discussion.

Itamar Gilad：哦，这是一个完全不同的讨论。

You had, Christina, the real expert over there so I doubt I can add more to that.

克里斯蒂娜，你是那里真正的专家，所以我怀疑我是否可以补充更多内容。

But it's true OKR is all part of it, but let's start with goals.

但确实 OKR 是其中的一部分，但让我们从目标开始。

What's our goals supposed to be?

我们的目标应该是什么？

Goals are supposed to paint the end state to define where we want to end up and the evidence will not guide you unless you know where you want to go, and in many companies what you have is goals at the top for revenue, market share, whatever it is, and then a bunch of siloed goals for each department.

目标应该描绘最终状态，以定义我们想要的最终目标，除非你知道自己想去哪里，否则证据不会指导你，在许多公司中，你拥有的最高目标是收入、市场份额等，然后是每个部门的一堆孤立的目标。

There's engineering goals, there's design goals, there's marketing goals, etc, and that actually pushes people into different vectors and it's really hard to decide.

有工程目标、有设计目标、​​有营销目标等等，这实际上将人们推向不同的方向，而且真的很难决定。

I would argue that in evidence guided companies, and you've worked for a few so probably you've seen this.

我认为，在证据引导的公司中，你曾为一些公司工作过，所以你可能已经看到了这一点。

They use models in order to construct overarching goals for the entire organization.

他们使用模型来构建整个组织的总体目标。

One of the models I show in the chapter about goals is the value exchange loop.

我在目标章节中展示的模型之一是价值交换循环。

Itamar Gilad: Where basically the organization is trying to deliver as much value as it can to the market and to capture as much value back, and by creating a feedback loop between these two you are actually able to grow very fast.

Itamar Gilad：基本上，组织正在努力向市场提供尽可能多的价值并夺回尽可能多的价值，通过在这两者之间创建反馈循环，您实际上能够非常快速地成长。

Now, I would argue that you want to measure both of these and to put a metric on each and the metric we usually use to measure value delivered is called the North Star metric.

现在，我认为您想要衡量这两个指标，并为每个指标制定一个指标，而我们通常用来衡量交付价值的指标称为北极星指标。

I know you wrote an article, a very good article about it.

我知道你写了一篇文章，一篇关于它的非常好的文章。

Itamar Gilad: And in it you listed dozens and dozens of companies, like leading companies and what they considered the North Star metric is super interesting.

Itamar Gilad：您在其中列出了数十家公司，例如领先公司，他们认为北极星指标非常有趣。

I would argue that what they told you is what is the most important metrics we measure?

我认为他们告诉你的是我们衡量的最重要的指标是什么？

What is the number one metric for us?

对我们来说第一大指标是什么？

But it's not what I call the North Star metric, the North Star metric measures how much value we create for the market.

但这不是我所说的北极星指标，北极星指标衡量的是我们为市场创造了多少价值。

For example, let's take WhatsApp.

以 WhatsApp 为例。

WhatsApp for a very long time measured messages sent because every message sent is a little incremental of value for the sender, the receiver, it's free, it's rich media, you can send it for anywhere in the world, compared to SMS that's huge value.

WhatsApp 长期以来一直在衡量发送的消息，因为发送的每条消息对于发送者和接收者来说都是一点点增值，它是免费的，它是富媒体，你可以将其发送到世界任何地方，与短信相比，它的价值巨大。

So if in year one we have a billion messages being sent in year two, two billion probably we doubled the amount of value.

因此，如果在第一年我们有十亿条消息在第二年发送，那么我们的价值可能会增加一倍，达到二十亿条。

In Airbnb, I think one of your key metrics or the real North Star metric was nights booked.

在爱彼迎，我认为关键指标之一或真正的北极星指标是预订天数。

I don't know if it was still the case while you were there?

不知道你在的时候还这样吗？

Itamar Gilad: And there are examples like this in Amplitude for example, they measure active learning users or weekly active learning users.

Itamar Gilad：Amplitude 中有这样的例子，例如，他们测量主动学习用户或每周主动学习用户。

Which are users that found in the tool some insight that was so important that they shared it with at least two other users and they consume it.

哪些用户在该工具中发现了一些非常重要的见解，以至于他们与至少两个其他用户分享并使用了它。

So it's a very powerful thing to point at this metric and say, "This is the most important metric combined with the value metric that we want to capture, revenue, market share, whatever it is." Once you have these two, you can further break them down into what I call metrics trees.

因此，针对这个指标并说：“这是最重要的指标，与我们想要捕获的价值指标（收入、市场份额等）相结合，这是一件非常有力的事情。”一旦有了这两个，您就可以将它们进一步分解为我所说的指标树。

So there's a metric three for the North Star metric and there's the metric three for the top KPI, the top business metric which you see here on the left side in blue and usually they overlap.

因此，北极星指标有一个指标三，顶级 KPI 也有指标三，您可以在左侧以蓝色看到顶级业务指标，并且通常它们会重叠。

So you might find in the middle some metrics that are super, super important because moving them actually moves the needle on everything else.

因此，您可能会在中间发现一些非常非常重要的指标，因为移动它们实际上会改变其他一切。

Lenny: Can you clarify again the difference between what you call this top KPI versus North Star metric?

Lenny：您能再次澄清一下您所说的最高 KPI 与北极星指标之间的区别吗？

Itamar Gilad: So the North Star metric is measuring how much value we're creating for the user, the core value that they're getting.

Itamar Gilad：北极星指标衡量的是我们为用户创造了多少价值，即他们获得的核心价值。

In this case this is some productivity suite, so this is number of documents created per month for example.

在本例中，这是一些生产力套件，因此这是每月创建的文档数量。

Because we think that every document created maybe it's a small document, I don't know.

因为我们认为创建的每个文档可能都是一个小文档，我不知道。

AI is in fashion now, is a little incremental value, so that's the number we're trying to grow.

人工智能现在很流行，它有一点增量价值，所以这就是我们正在努力增长的数字。

The top KPI is what we expect to get, it should be revenue or profit.

最上面的KPI是我们期望得到的，它应该是收入或者利润。

Lenny: I see, this is the value exchange.

莱尼：我明白了，这就是价值交换。

I see, one is what users are getting, one is what you're getting back from them.

我明白了，一是用户得到的，一是你从他们那里得到的。

Lenny: Basically how the business is benefiting.

莱尼：基本上是企业如何受益。

Awesome.

惊人的。

I think this is a really important concept, the metric tree.

我认为这是一个非常重要的概念，度量树。

I think a lot of people think they have something like this in mind where they're just like, "Cool, here's our North Star Metric, here's the levers and things that we can work on to move that." But I think actually mapping it out the way you have it here where it kind of goes layers and layers deep to all of the different variables that impact this metric.

我想很多人都认为他们心里有这样的想法，他们只是想，“酷，这是我们的北极星指标，这是我们可以努力推动这一目标的杠杆和事情。”但我认为实际上按照这里的方式进行映射，它会一层又一层地深入到影响该指标的所有不同变量。

Not only is it a way to think about impact and goals and things like that, but also helps you estimate the impact of the experiment you're potentially thinking about running.

它不仅是一种思考影响和目标以及类似事物的方法，而且还可以帮助您估计您可能考虑进行的实验的影响。

So if you're going to work on something at the bottom here like activation rate, say you move that 10%.

因此，如果您要处理底部的某些问题（例如激活率），请假设您将其移动 10%。

How much is that going to impact this global metric?

这会对这一全球指标产生多大影响？

It's probably a very small amount.

这可能是一个非常小的数量。

Itamar Gilad: This is a very important one and we'll talk about impact assessment shortly, this helps with it.

Itamar Gilad：这是一个非常重要的问题，我们很快就会讨论影响评估，这会有所帮助。

It also helps with alignment because the entire organization is trying to move these two metrics, it's the two sides of our mission essentially.

它还有助于协调一致，因为整个组织都在努力移动这两个指标，这本质上是我们使命的两个方面。

We have the mission that's the top objective of the company and these are the two top most key results if you like, the top most things.

我们的使命是公司的首要目标，如果你愿意的话，这是两个最重要的结果，最重要的事情。

So when you go and work with another team and you say, "Hey, why don't you work on my project?" They might say, "This idea actually might move the North Star metric model in your idea." And that helps you guys align and I've seen cases where team B put aside their own ideas to jump on the ideas of team A, because of this model.

因此，当您去与另一个团队合作时，您会说：“嘿，为什么不为我的项目工作呢？”他们可能会说，“这个想法实际上可能会改变你想法中的北极星度量模型。”这可以帮助你们协调一致，而且我见过 B 团队搁置自己的想法而跳到 A 团队的想法的案例，因为这个模型。

It also creates an opportunity to give some sub metrics to teams to own on an ongoing basis, so it creates a little sense of ownership as well and mission within the tree.

它还创造了一个机会，让团队能够持续拥有一些子指标，因此它也在树中创造了一点主人翁感和使命感。

Lenny: It also helps you figure out what teams you should have, which teams have the biggest potential to impact the metric.

Lenny：它还可以帮助您找出应该拥有哪些团队，哪些团队最有可能影响指标。

Itamar Gilad: Another thing that happens in a lot of organizations, the team topology reflects the structure of the software or some hierarchical model where we want to organize the organization in a particular way.

Itamar Gilad：在许多组织中发生的另一件事是，团队拓扑反映了软件的结构或我们希望以特定方式组织组织的某种层次模型。

But if you start with a metrics tree, you can try to arrange the topology around goals and sometimes you need to readjust.

但如果您从指标树开始，您可以尝试围绕目标排列拓扑，有时您需要重新调整。

It's not a constant reorg but from time to time you will realize the goals have changed and we need to reorganize, so the tree helps visualize that as well.

这不是一个持续的重组，但有时你会意识到目标已经改变，我们需要重新组织，所以树也有助于可视化。

Lenny: I think for people that are listening to this and thinking about this, I think the simplest way to even think about this is basically there's a math formula that equals your North Star metric or your revenue or whatever you're trying to do and if you don't have some ideally really clear sense of what that math formula is you should work on that.

莱尼：我认为对于那些正在听这个并思考这个问题的人来说，我认为思考这个问题的最简单方法基本上是有一个数学公式，它等于你的北极星指标或你的收入或任何你想要做的事情，如果你对这个数学公式是什么没有一些理想的清晰认识，你应该研究它。

Because that will inform so much of how you think about where to invest, what teams to have, where to invest more resources, less resources.

因为这将告诉你如何思考在哪里投资、拥有什么团队、在哪里投资更多资源、更少资源。

Lenny: Imagine a place where you can find all your potential customers and get your message in front of them in a cost-efficient way.

Lenny：想象一个地方，您可以找到所有潜在客户，并以经济高效的方式向他们传达您的信息。

If you're a B2B business, that place exists and it's called LinkedIn.

如果您从事 B2B 业务，那么这个地方就存在，它的名字叫 LinkedIn。

LinkedIn Ads allows you to build the right relationships, drive results, and reach your customers in a respectful environment.

LinkedIn 广告可让您建立正确的关系、推动成果并在相互尊重的环境中接触您的客户。

Two of my portfolio companies Webflow and Census are LinkedIn success stories.

我投资的两家公司 Webflow 和 Census 都是 LinkedIn 的成功案例。

Census had a 10x increase in pipeline with the LinkedIn startup team, for Webflow after ramping up on LinkedIn in Q4 they had the highest marketing source revenue quarter to date.

Census 与 LinkedIn 初创团队的合作渠道增加了 10 倍，Webflow 在第四季度在 LinkedIn 上加大力度后，获得了迄今为止季度最高的营销来源收入。

With LinkedIn Ads, you'll have direct access to and can build relationships with decision makers including 950 million members, 180 million senior execs and over 10 million C-level executives.

借助 LinkedIn Ads，您将可以直接接触决策者并与其建立关系，其中包括 9.5 亿会员、1.8 亿高级管理人员和超过 1000 万 C 级管理人员。

You'll be able to drive results with targeting and measurement tools built specifically for B2B.

您将能够利用专为 B2B 构建的定位和衡量工具来推动结果。

In tech LinkedIn, generated two to five X higher return on ad spend than any other social media platforms.

在科技领英上，其广告支出回报率比任何其他社交媒体平台高出两到五倍。

Audiences on LinkedIn, have two times the buying power of the average web audience and you'll work with a partner who respects the B2B world you operate in.

LinkedIn 上的受众的购买力是普通网络受众的两倍，您将与尊重您所在 B2B 世界的合作伙伴合作。

Lenny: Make B2B marketing everything it can be and get $100 credit on your next campaign, just go to linkedin.com/podlenny to claim your credit.

Lenny：尽一切可能进行 B2B 营销，并在下一次营销活动中获得 100 美元积分，只需前往 linkedin.com/podlenny 领取积分即可。

That's linkedin.com/podlenny, terms and conditions apply.

这是 linkedin.com/podlenny，适用条款和条件。

Okay.

好的。

So metrics trees, what comes next?

那么指标树，接下来会发生什么？

Itamar Gilad: All right.

伊塔玛·吉拉德：好的。

So next we need to go to the ideas layer and the ideas layer is there to help us sort through the many ideas we might encounter and they may come from as you said the founders, the managers, the stakeholders, from the team, from research, from competitors.

因此，接下来我们需要进入想法层，想法层可以帮助我们整理可能遇到的许多想法，它们可能来自你所说的创始人、经理、利益相关者、团队、研究、竞争对手。

We're flooded with ideas, and what usually happens inside organization is some sort of battle of opinions or some sort of politics sometimes or highest paid person's opinion.

我们充满了想法，组织内部通常发生的是某种意见之争，有时是某种政治，或者是最高薪者的意见。

You had, Ronny Kohavi, who invented this term in your show.

Ronny Kohavi，你在你的节目中发明了这个术语。

What doesn't happen is very rational, logical decisions these are the best ideas, because it's really, really hard to predict honestly.

不会发生的是非常理性、合乎逻辑的决定，这些都是最好的想法，因为它真的很难诚实地预测。

There is so much uncertainty in the needs of the users, in the changes in the market, in our technology, in our product, in our own organization.

用户的需求、市场的变化、我们的技术、我们的产品、我们自己的组织都存在很多不确定性。

It's almost impossible to say this idea is going to be the best, but we do say this because we have cognitive biases that kind of convince us that this idea is far superior to anything else and it's definitely the right choice.

几乎不可能说这个想法是最好的，但我们确实这么说，因为我们有认知偏见，这让我们相信这个想法远远优于其他任何想法，而且它绝对是正确的选择。

Itamar Gilad: In order to avoid this, what we want to do is to evaluate the ideas in a much more objective and consistent and transparent way.

Itamar Gilad：为了避免这种情况，我们想要做的是以更加客观、一致和透明的方式评估这些想法。

In the book I suggest using ICE, impact, confidence and ease.

在书中我建议使用ICE，影响、自信和轻松。

I think I have a slide coming on this.

我想我有一张幻灯片来讨论这个问题。

So impact, confidence and ease which is basically a way to assign three values to each idea.

因此，影响力、信心和轻松感基本上是为每个想法分配三个价值的一种方式。

The impact tries to assess how much impact it'll have on the goals and that's why it's so important that we have very clear goals and not many.

影响试图评估它对目标的影响有多大，这就是为什么它如此重要，我们有非常明确的目标，但目标不多。

How we are measuring the ideas on the North Star metric, on the top business KPI, on a local metric of the team.

我们如何根据北极星指标、顶级业务 KPI、团队的本地指标来衡量想法。

Whatever it is, let's be clear about it and then let's evaluate the ideas against this thing.

不管它是什么，让我们弄清楚它，然后让我们根据这个东西评估我们的想法。

Ease, is basically the opposite of effort.

轻松，基本上与努力相反。

How easy or hard it's going to be, but both of those are guesstimates, both of those are things we need to estimate.

这会是多么容易或困难，但这两者都是猜测，都是我们需要估计的事情。

I would argue that just by breaking the question to these two questions we usually have a slightly better discussion than just my idea is better than yours.

我认为，只要将问题分解为这两个问题，我们通常会进行比我的想法比你的更好更好的讨论。

Itamar Gilad: But then there's the third element which is confidence, which tries to assess how sure are we or should we be about our first guesstimates about the impact and the ease.

伊塔马尔·吉拉德：但是还有第三个要素，那就是信心，它试图评估我们对影响和难易程度的初步猜测有多大把握或应该有多大把握。

Lenny: It's interesting you use the word ease, because I think it's usually effort.

莱尼：有趣的是你使用了“轻松”这个词，因为我认为这通常是指努力。

You kind of make it positive, is that an intentional tweak you made?

你让它变得积极，这是你故意做出的调整吗？

Itamar Gilad: I'm using the definitions of, Sean Ellis.

Itamar Gilad：我使用的是 Sean Ellis 的定义。

Sean, invented ICE.

肖恩，发明了ICE。

You know Sean, I don't know if you've had him yet?

你认识肖恩，不知道你有没有拥有过他？

But he's-

但他是——

Itamar Gilad: Yeah.

伊塔玛·吉拉德：是的。

For the people who don't know him, Sean, is amazing.

对于不认识他的人来说，肖恩是个了不起的人。

He's like one of the fathers of the growth movement, he coined the term growth hacking and he popularized the concept of product market fit.

他就像增长运动之父之一，他创造了“增长黑客”一词，并普及了产品市场契合度的概念。

Itamar Gilad: He created ICE, he created a bunch of things that we use in product that we don't even know.

Itamar Gilad：他创建了 ICE，他创建了一堆我们在产品中使用的东西，但我们甚至不知道。

Lenny: Wow, I didn't know he came up with ICE.

莱尼：哇，我不知道他发明了 ICE。

Okay, cool.

好吧，酷。

So the original version of ICE is ease instead of effort.

所以 ICE 的原始版本是轻松而不是努力。

Itamar Gilad: A lot of your viewers are wondering where's the R because there's another variant of this culture.

Itamar Gilad：很多观众都想知道 R 在哪里，因为这种文化还有另一种变体。

RICE, where there's rich as well.

稻米，那里也有钱。

I prefer ICE because I prefer to fold the rich into the I for various reasons but both are valid, both are equivalent in a sense.

我更喜欢ICE，因为出于各种原因我更喜欢将富人折叠到I中，但两者都是有效的，两者在某种意义上是等价的。

Lenny: I'm in your boat, that's exactly how I think about it.

莱尼：我和你一样，这正是我的想法。

I think people over complicate this stuff and try to get so many math formulas involved with estimating impact, and I feel like these are just simple heuristics to kind of bubble the best ideas to the top.

我认为人们把这件事过于复杂化，并试图让如此多的数学公式参与估计影响，我觉得这些只是简单的启发式方法，可以将最好的想法冒出来。

It doesn't have to be a perfect estimate of impact and confidence and all those things, so I think the simpler is better and it always ends up being a spreadsheet.

它不一定是对影响力和信心以及所有这些事情的完美估计，所以我认为越简单越好，它最终总是一个电子表格。

People always have these tools to estimate these things but it's like a spreadsheet, Google Sheets.

人们总是有这些工具来估计这些事情，但它就像电子表格，Google Sheets。

Great.

伟大的。

Itamar Gilad: So yeah, you're actually leading me to my next point.

Itamar Gilad：是的，你实际上正在引导我进入下一个观点。

So when you come to estimate impact you will realize it's the hardest part.

因此，当您评估影响时，您会意识到这是最难的部分。

So sometimes it's just a gut feeling and it's a guess and sometimes it's based on some spreadsheet or some analysis and the back of envelope calculation you've done and I think that's legitimate.

所以有时这只是一种直觉和猜测，有时它是基于一些电子表格或一些分析以及你所做的粗略计算，我认为这是合理的。

Sometimes these things do show you some things you didn't think of and sometimes the best case it's based on tests.

有时这些东西确实会向您展示一些您没有想到的东西，有时最好的情况是基于测试。

You actually tested it, you interviewed 12 customers, you show them the thing and out of those only one actually liked it.

你实际上测试了它，你采访了 12 位客户，你向他们展示了这个东西，而其中只有一个人真正喜欢它。

You should reduce your impact based on that usually, or you do other types of tests.

您应该根据通常情况减少影响，或者进行其他类型的测试。

We'll talk about testing in a second.

我们稍后会讨论测试。

What happens is that people tend to just go with gut instinct and then give themselves a high confidence.

发生的情况是，人们往往只是凭直觉行事，然后给自己很高的信心。

They say it's an eight and I'm pretty convinced, so it's eight for confidence and I found this a bit disturbing because it kind of subverts the whole system.

他们说这是八，我非常确信，所以这是八的信心，我发现这有点令人不安，因为它有点颠覆整个系统。

Itamar Gilad: So I wanted to help people realize when they have strong evidence in support of their guesses and when it's weak evidence, how to calculate confidence in a sense.

Itamar Gilad：所以我想帮助人们认识到当他们有强有力的证据支持他们的猜测时，以及当证据很弱时，如何在某种意义上计算置信度。

For that I created a tool called the confidence meter, which you can see here this colorful thing and should I go and explain it?

为此，我创建了一个名为“置信度计”的工具，您可以在这里看到这个丰富多彩的东西，我应该去解释一下吗？

Lenny: Yeah, let's do it.

莱尼：是啊，我们就这么做吧。

And then again, if you're just listening to this you can check this out on YouTube and you can see the actual slide.

再说一次，如果您只是听这个，您可以在 YouTube 上查看，您可以看到实际的幻灯片。

Itamar Gilad: All right, awesome.

Itamar Gilad：好的，太棒了。

So basically I constructed it a bit like a thermo meter.

所以基本上我把它构造得有点像温度计。

It goes from very low confidence which is the blue area or the upper right, all the way to high confidence which is the red area and you can see the numbers going from zero to 10.

它从非常低的置信度（蓝色区域或右上角）一直到高置信度（红色区域），您可以看到数字从 0 到 10。

Where zero is very low confidence, we don't know basically anything we're just guessing in the dark and 10 is full confidence.

0 表示置信度非常低，我们基本上不知道我们只是在黑暗中猜测的任何内容，而 10 表示完全置信度。

You know for sure this thing is a success, no doubt about it and across the circle I put various classes of evidence you might find along the way.

你肯定知道这件事是成功的，毫无疑问，我在整个圈子里放了你可能会发现的各种证据。

So for example, starting at the top right, all of these blue areas about opinions.

例如，从右上角开始，所有这些关于意见的蓝色区域。

It could be your own self-confidence in the idea, your self conviction, you feel it's a great idea.

这可能是你对这个想法的自信，你的自信，你觉得这是一个很棒的想法。

Guess what?

你猜怎么着？

Behind every terrible idea that was ever someone thought it was great, that gives you 0.01 out of 10.

在每一个可怕的想法背后，有人认为它很棒，这给你 0.01 分（满分 10 分）。

Maybe you created a shiny pitch deck or a six-page document that explains in detail why this is a great idea.

也许您创建了一个闪亮的宣传材料或一份六页的文档，详细解释了为什么这是一个好主意。

Slightly harder to do but still very low confidence, maybe you connected it to some theme, it's about the blockchain...

稍微难一点，但信心仍然很低，也许你将它与某个主题联系起来，这是关于区块链的......

Itamar Gilad: Well sorry, the blockchain is out of fashion.

Itamar Gilad：抱歉，区块链已经过时了。

What's hot right now?

现在什么最热？

Itamar Gilad: Exactly, AI.

Itamar Gilad：没错，人工智能。

It's about AI, that makes it a good idea?

这是关于人工智能的，这使它成为一个好主意？

Absolutely not.

绝对不是。

Or the strategy of the company, that's another thematic support.

或者公司的战略，这是另一个主题支持。

Thousands and thousands of terrible ideas are being implemented right now as we speak based on these themes.

当我们基于这些主题发言时，成千上万的可怕想法正在被实施。

So all these things combined can give you a maximum 0.1 out of 10 according to the tool, if you follow it then we move into slightly harder tests.

因此，根据该工具，所有这些因素结合起来最多可以给你 0.1 分（满分 10 分），如果你遵循它，那么我们将进入稍微困难的测试。

One is reviewing it with your colleagues, your managers, your stakeholders the idea.

一是与你的同事、经理、利益相关者一起审查这个想法。

They don't know it either, they don't have a crystal ball, they're usually not the users, they cannot predict.

他们也不知道，他们没有水晶球，他们通常不是用户，他们无法预测。

But they can evaluate it in a slightly more objective way and maybe find flaws in your idea.

但他们可以以稍微更客观的方式对其进行评估，并可能发现你的想法中的缺陷。

On the other hand groups tend to have biases too, politics group thing.

另一方面，团体也往往有偏见，政治团体的事情。

So groups can actually arrive sometimes with worse decisions than individuals, there's some research to that.

因此，团队实际上有时会做出比个人更糟糕的决定，对此有一些研究。

Next, our estimates and plans.

接下来是我们的估计和计划。

So you may do some sort of back of the envelope calculation or your colleagues might go out and try to evaluate the ease a little bit better.

因此，您可能会进行某种粗略的计算，或者您的同事可能会出去尝试更好地评估易用性。

Itamar Gilad: That gives you a little bit more confidence, but still we're at the level of guesswork at this point.

Itamar Gilad：这让您更有信心，但目前我们仍处于猜测水平。

Next we're moving to data and data could be anecdotal.

接下来我们转向数据，数据可能是轶事。

So you find a few data points dotted across your data or you talk to a handful of customers or maybe one competitor has that same idea.

因此，您会发现数据中散布着一些数据点，或者您与少数客户交谈，或者也许一个竞争对手也有同样的想法。

In many companies I meet, if the leading competitor has this feature and we think it's a good idea validation is done.

在我遇到的许多公司中，如果主要竞争对手具有此功能，并且我们认为这是一个好主意，那么就会进行验证。

Let's launch it, that's it.

让我们启动它，就是这样。

It's a great idea, we need to do it.

这是个好主意，我们需要这样做。

It never works honestly, you should not assume that your competitor actually knows what they're doing anymore than you do.

它永远不会诚实地起作用，你不应该假设你的竞争对手实际上比你更知道他们在做什么。

Data could be also what I call market data.

数据也可以是我所说的市场数据。

That comes from surveys, from assessing a lot of your data by doing a deep competitive analysis and there are other methods where you create a larger dataset and you contrast your idea against it.

这来自调查，通过进行深入的竞争分析来评估大量数据，还有其他方法可以创建更大的数据集，并将你的想法与它进行对比。

Finally, to gain medium and high confidence you really need to build your idea and test it and that's where the red area is.

最后，要获得中度和高度的信心，您确实需要构建您的想法并对其进行测试，这就是红色区域所在的位置。

Itamar Gilad: So there's various forms of tests, we'll talk about them if we have time and they give you various levels of confidence.

Itamar Gilad：所以有各种形式的测试，如果有时间我们会讨论它们，它们会给你不同程度的信心。

Lenny: Awesome, this is a very cool visual.

莱尼：太棒了，这是一个非常酷的视觉效果。

We'll link to a image of this in the show notes too if people want to check it out.

如果人们想查看的话，我们也会在演出笔记中链接到该图片。

I think what's awesome about this is you could just use this as a little tool on your team of just like where are we along the spectrum?

我认为这最棒的是你可以在你的团队中使用它作为一个小工具，就像我们处于哪个阶段？

We think the impact of this is very high.

我们认为这的影响非常大。

But we're probably in this blue area of confidence and so let's just make sure we understand that and it's really clear language to help people understand.

但我们可能处于这个蓝色的信心区域，所以让我们确保我们理解这一点，并且这是非常清晰的语言来帮助人们理解。

I see if we had this, it'd be a lot more confident.

我想如果我们有这个，就会更有信心。

Itamar Gilad: So you can also tie your investment into the idea based on the level of confidence you had found essentially, so early on you want to do the cheap stuff just to gain more confidence and then you can go and invest more.

Itamar Gilad：所以你也可以根据你基本上发现的信心水平将你的投资与这个想法联系起来，所以一开始你想做便宜的事情只是为了获得更多信心，然后你可以去投资更多。

If it's a really cheap idea, you can jump to a high confidence idea, you can test, you can do an AB experiment.

如果这是一个非常便宜的想法，你可以跳到一个高置信度的想法，你可以测试，你可以做AB实验。

Early adopter program, whatever it is and then launch it.

早期采用者计划，无论是什么，然后启动它。

Some ideas you don't need to test, sometimes the expert opinion is enough.

有些想法不需要测试，有时专家的意见就足够了。

If you're just changing the order of the settings, no one sees this or no one will be impacted.

如果您只是更改设置的顺序，则没有人会看到这一点，或者没有人会受到影响。

The risk is low, you can launch it without testing.

风险低，无需测试即可上线。

So part of the trick is also knowing when to stop, not just trying to force your way all the way up when you don't have to.

因此，技巧的一部分还在于知道何时停止，而不仅仅是在不必要时试图强行一路向上。

Lenny: That's a really important point.

莱尼：这是非常重要的一点。

The other important point here is just a big part of a PM's job is to say no and to stop stupid shit from happening and this is an awesome tool to help you do that.

这里的另一个重要点是产品经理工作的一个重要部分，就是说不并阻止愚蠢的事情发生，这是一个很棒的工具可以帮助你做到这一点。

To be like, okay, here's this idea you have, just like let's just be real, how confident are we in this?

就像，好吧，这就是你的想法，就像让我们说实话，我们对此有多大信心？

And, okay, it's going to take us three months to do this.

好吧，我们需要三个月的时间才能做到这一点。

Maybe we should think about something different, maybe we should work up the confidence meter before we actually commit to this.

也许我们应该考虑一些不同的事情，也许我们应该在真正做出承诺之前先建立信心表。

Itamar Gilad: Yeah.

伊塔玛·吉拉德：是的。

This is a real world usage that I hear about a lot, some people use this to kind of do...

这是我经常听到的现实世界用法，有些人用它来做......

An objective way to say no and gently.

一种客观且温和地说“不”的方式。

Or to say we'll think about it but look at these other ideas we have and how their impacting is and confidence stack up.

或者说我们会考虑它，但看看我们拥有的其他想法以及它们的影响和信心如何叠加。

Lenny: Classic PM move, just like that was a great idea but what about this better idea?

Lenny：经典的 PM 举措，就像这是一个好主意，但是这个更好的主意怎么样？

Coming back to something that we talked a bit about at the beginning, say you have a founder who's actually very smart and experienced.

回到我们一开始谈到的事情，假设你有一位实际上非常聪明且经验丰富的创始人。

Say even at a startup where you don't really have the time to build tons of evidence for ideas.

甚至在一家初创公司，你也没有时间为想法建立大量证据。

Do you have a different perspective on how much time to spend building confidence in ideas versus just like cool, they actually have really good ideas let's just see what happens?

对于花多少时间来建立对想法的信心与就像酷一样，他们实际上有非常好的想法让我们看看会发生什么，你有不同的看法吗？

Itamar Gilad: So there's always like a trade-off between speed of delivery and speed of discovery, and that actually leads to the next layer of how do we combine the two?

Itamar Gilad：因此，交付速度和发现速度之间总是需要权衡，这实际上导致了我们如何将两者结合起来的下一层？

Because people tend to think it's an either/or.

因为人们倾向于认为这是一个非此即彼的问题。

Either we are building very fast or we are learning and then we're building very slow, but I think we're using the wrong metric.

要么我们构建得非常快，要么我们正在学习，然后我们构建得非常慢，但我认为我们使用了错误的指标。

The metric is not how fast can we get the bits into production, when there's a lot of uncertainty and we all face uncertainty and startup especially.

当存在很多不确定性并且我们都面临着不确定性和启动时，衡量标准不是我们能以多快的速度将这些比特投入生产。

It's not about getting the bits to production, it's about getting the right bits to production.

这不是为了将这些部分投入生产，而是为了将正确的部分投入生产。

It's about creating the outcomes that you need, the impact, and so it's about time to outcomes and I would argue that the evidence guided method is far more impactful.

这是关于创造你需要的结果和影响，所以现在是时候取得结果了，我认为证据引导的方法更有影响力。

It's far faster, it's far more resource efficient than the opinion-based method.

与基于意见的方法相比，它速度更快，资源效率更高。

Because opinion-based methods tend to waste a lot more of your resources, building the wrong things or discovering, learning too late.

因为基于意见的方法往往会浪费更多的资源，构建错误的东西或发现、学习太晚。

Well, evidence guided helps you learn earlier.

嗯，证据引导可以帮助你更早地学习。

Itamar Gilad: Plus it is a fallacy that if you learn you don't build, good teams know how to do both at the same time and that's actually what the steps layer is meant to teach you or to help you do.

Itamar Gilad：另外，如果你知道自己不构建，那么优秀的团队知道如何同时做这两件事，这实际上是步骤层旨在教你或帮助你做的事情，这是一个谬论。

Lenny: Awesome.

莱尼：太棒了。

So maybe just to close off that loop, say someone listening is at a bigger company, say Netflix versus a series A, series B or startup.

因此，也许只是为了结束这个循环，假设有人在一家更大的公司聆听，比如 Netflix 与 A 系列、B 系列或初创公司。

Is there something you'd recommend about them approaching this differently?

对于他们以不同的方式处理这个问题，您有什么建议吗？

Any kind of guidance there of just how to take what you're sharing differently if you're a different source of companies like that?

如果你是这样的公司的不同来源，有什么指导可以指导你如何以不同的方式分享你所分享的内容吗？

Itamar Gilad: Absolutely.

伊塔玛·吉拉德：当然。

I think the concept we talked about of the North Star metric, the value created versus the value captured is very important in every company.

我认为我们讨论的北极星指标的概念，即创造的价值与获取的价值对每家公司都非常重要。

Building your entire metrics trees, maybe overkill, doing heavy weighted OKRs may be overkill for early stage.

构建整个指标树，可能有点过分，在早期阶段进行重量级的 OKR 可能有点过头了。

Early stage companies even don't know how they create value, so they need to iterate and their goals is really to find product market fit.

早期公司甚至不知道自己如何创造价值，因此他们需要迭代，他们的目标实际上是找到适合市场的产品。

Beyond that, what happens is that you need to start building your business model.

除此之外，您需要开始构建您的商业模式。

So that's your goal and you iterate towards that and you need to put metrics on that and then when you move into scale, you need to try to create order because when you scale up...

这就是你的目标，你要迭代这个目标，你需要对其进行衡量，然后当你扩大规模时，你需要尝试创建订单，因为当你扩大规模时......

And all of this is covered in the book, there's a special chapter just about these questions.

所有这些都在本书中涵盖，有一个专门的章节专门讨论这些问题。

When you scale up, you get a lot of people and a lot of money and everything is happening at the same time.

当你扩大规模时，你会得到很多人和很多钱，所有事情都会同时发生。

So there you need a order of evaluating ideas in a very systematic way.

因此，您需要以非常系统的方式评估想法的顺序。

In a company like Netflix, by the way I don't know if they need this specific method.

在Netflix这样的公司，顺便说一句我不知道他们是否需要这种特定的方法。

They're very-

他们非常——

Lenny: Yeah, maybe that was a bad example.

莱尼：是的，也许这是一个坏例子。

They're probably doing things pretty well.

他们可能做得很好。

Itamar Gilad: One thing I discovered by the way, there's two types of companies that really benefit from this technique.

Itamar Gilad：顺便说一句，我发现有两类公司真正从这项技术中受益。

One is those companies that are kind of emerging into modern product development.

一类是那些正在进入现代产品开发领域的公司。

They have product teams, they have product managers, they have OKRs, they're starting to do Agile.

他们有产品团队，有产品经理，有 OKR，他们开始实施敏捷。

But they're starting to do experimentation, but they're struggling to put it all together.

但他们已经开始进行实验，但他们正在努力将它们整合在一起。

Every CPO is building their own little framework and the other type is those companies that used to be evidence guided and they regressed and that happens way too often.

每个 CPO 都在构建自己的小框架，另一类是那些过去以证据为导向的公司，但他们却出现了倒退，这种情况经常发生。

Change of management, change of culture, and then all of a sudden they need to rediscover, to rekindle that spirit that was lost along Google+.

管理层的变化，文化的变化，然后突然间他们需要重新发现，重新点燃 Google+ 失去的精神。

So some of the people that actually respond to the strongest are actually surprisingly in these companies.

因此，在这些公司中，一些真正对最强烈反应的人实际上是令人惊讶的。

Lenny: What I love about your frameworks and kind of all these things we're talking about is these are just a...

Lenny：我喜欢你们的框架以及我们正在谈论的所有这些事情，因为这些只是......

You can almost think of them as a grab bag set of tools to make you more evidence guided as a company.

您几乎可以将它们视为一套工具，让您作为一家公司获得更多证据指导。

You could start with thinking about the confidence meter, you could start using ICE more.

您可以从考虑置信度计开始，您可以开始更多地使用 ICE。

You could start using the metrics tree and all these things just push you closer and closer to being more evidence guided, you don't have to adopt this whole thing all at once.

您可以开始使用指标树，所有这些事情只会让您越来越接近更多证据的指导，您不必立即采用整个事情。

Itamar Gilad: Absolutely.

伊塔玛·吉拉德：当然。

I would recommend that you don't try because if the transformation is way too big, you will get fatigued and you will just create a lot of process for a lot of people and you would not see the results and after a quarter you'll give up.

我建议你不要尝试，因为如果转型太大，你会感到疲劳，你只会为很多人创建很多流程，而你不会看到结果，一个季度后你就会放弃。

So exactly what you suggested is the right approach.

所以你所建议的正是正确的方法。

Lenny: What would be the first thing you'd suggest if people were trying to move closer to being less opinion oriented and more evidence-based?

莱尼：如果人们试图减少以意见为导向，多以证据为基础，你会建议的第一件事是什么？

Which of these frameworks or models would you recommend first?

您首先推荐以下哪些框架或模型？

Itamar Gilad: I recommend that they discuss internally where is the biggest problem that they're facing.

Itamar Gilad：我建议他们在内部讨论他们面临的最大问题在哪里。

If the goals are unclear, there's misalignment, we keep chasing the wrong things, start at the goals layer.

如果目标不明确，就会出现偏差，我们就会不断追求错误的东西，从目标层开始。

Try to establish your North Star metric, your top business metric, your metrics trees, start assigning teams with their own area of responsibility.

尝试建立你的北极星指标、你的首要业务指标、你的指标树，开始为团队分配他们自己的职责范围。

If you're spending a lot of time in debates and you're constantly fighting and changing your mind.

如果您花费大量时间进行辩论并且不断地争吵并改变主意。

Start with the ideas there and establish impact is confidence or whatever prioritization model you like, but involve evidence in it.

从那里的想法开始，建立影响力是信心或任何你喜欢的优先级模型，但要在其中包含证据。

I think the confidence meter is a good tool to use irrespective.

我认为无论如何，置信度都是一个很好的工具。

If you're building too much and you're not learning enough, start adopting the steps layer which we haven't seen yet and if your team is very disengaged.

如果您构建了太多内容并且学习得不够，那么如果您的团队非常不积极参与，请开始采用我们尚未看到的步骤层。

You have one of these teams where the developers are very into Agile, very into quality, very into launching things, start working on the tasks there.

你有一个这样的团队，其中的开发人员非常热衷于敏捷，非常热衷于质量，非常热衷于发布东西，开始在那里完成任务。

Lenny: Awesome.

莱尼：太棒了。

Okay, let's keep going.

好吧，我们继续吧。

Itamar Gilad: All right, so steps.

Itamar Gilad：好的，开始吧。

Steps are about kind of helping us learn and build at the same time as we said and one of the patterns I see is that organizations don't know that they can actually learn at a much lower cost.

正如我们所说，步骤是为了帮助我们同时学习和构建，我看到的模式之一是组织不知道他们实际上可以以更低的成本进行学习。

They believe they need to build this elaborate MVP which is not minimal in any way and then launch it and then they will discover it and basically it's what we used to call beta 20 years ago but just with a different name.

他们相信他们需要构建这个精心设计的 MVP，它在任何方面都不是最小的，然后启动它，然后他们会发现它，基本上它就是我们 20 年前所说的 beta 版本，只是名称不同。

What I'm trying to do here in the steps layer is to help companies realize there's a gamut of ways to validate your ideas or more specifically to validate the assumptions in your idea and I created a little model for this, it's called after assessment fact finding, tests, experiments and release results.

我在步骤层中尝试做的事情是帮助公司认识到有多种方法可以验证您的想法，或者更具体地说，可以验证您的想法中的假设，我为此创建了一个小模型，在评估事实发现、测试、实验和发布结果之后调用它。

But again, it's just putting together things that much smarter people invented.

但同样，它只是将更聪明的人发明的东西放在一起。

So in assessment you have very easy things, things that don't require a lot of work.

因此，在评估中，你会遇到非常简单的事情，不需要做很多工作。

You check if it aligns with the goals, this idea that you have in your hand.

你检查它是否与你手中的目标、这个想法相符。

Itamar Gilad: You do maybe some business modeling, you do ICE analysis, you do Assumption Mapping which is great tool by, David J.

Itamar Gilad：你可能会做一些业务建模，你会做 ICE 分析，你会做假设映射，这是 David J. 的很棒的工具。

Blend, or you talk to your stakeholders one-on-one just to see if there are any risks, etc. These are usually not expensive things and they can teach you an awful lot about the impact and the ease of your idea.

混合，或者你与利益相关者进行一对一的交谈，看看是否存在任何风险等。这些通常不是昂贵的东西，它们可以教会你很多关于你的想法的影响和难易程度。

The next step is to dig data and usually that goes hand in hand with this.

下一步是挖掘数据，通常与此同时进行。

So you can find data in your data analysis through surveys, through competitive analysis, through user interviews and through field research, observing your users.

因此，您可以通过调查、竞争分析、用户访谈和实地研究来观察用户，从而在数据分析中找到数据。

Obviously these last two are pretty expensive, so it's often good not to wait until you have the idea and then start doing your research.

显然，最后两种方法相当昂贵，因此最好不要等到有了想法才开始进行研究。

It's best to keep doing your research ongoing and then you have some sort of data to lie on and to compare your idea against.

最好继续进行你的研究，然后你就有了一些数据可以依据并与你的想法进行比较。

But until now we didn't build anything, now you're ready to start testing, building versions of the product and putting them in front of users and measuring the results.

但到目前为止我们还没有构建任何东西，现在您已经准备好开始测试，构建产品的版本并将它们放在用户面前并测量结果。

But initially you don't build anything, you fake it.

但最初你不建造任何东西，你只是伪造它。

Itamar Gilad: You do a fake door test, you do a smoke test, Wizard of Oz test, a concierge test, usability test.

Itamar Gilad：你做了假门测试，你做了烟雾测试，绿野仙踪测试，礼宾测试，可用性测试。

We used a lot of those in the tabbed inbox by the way, one of the first early versions was actually we showed the tabbed inbox working to people.

顺便说一句，我们在选项卡式收件箱中使用了很多这些功能，第一个早期版本实际上是我们向人们展示了选项卡式收件箱的工作原理。

But it wasn't really Gmail, it was just a facade of HTML and behind the scenes and according to the permissions that the users gave us.

但它并不是真正的 Gmail，它只是 HTML 的一个外观，在幕后，根据用户授予我们的权限。

Some of us moved just the subject and the sender into the right place.

我们中的一些人只是将主题和发件人移到了正确的位置。

So initially the interviewer distracted them and then showed them their inbox and in it the top 50 messages were sorted to the right place, more or less if we got it right and people were like, "Wow, this is actually very cool." And that gave us a lot of evidence.

因此，最初面试官分散了他们的注意力，然后向他们展示了收件箱，其中前 50 条消息被分类到正确的位置，或多或少，如果我们做对了，人们会说，“哇，这实际上非常酷。”这给了我们很多证据。

Lenny: That's an awesome story.

莱尼：那是一个很棒的故事。

So that was in the user research, it wasn't rolled out to people?

那么这是在用户研究中，没有向人们推广？

It was a manual individual?

这是一个手动个体？

Itamar Gilad: There wasn't a single line of code written, this was just cooked up by the researchers and our designers.

Itamar Gilad：没有编写任何代码，这只是研究人员和我们的设计师炮制出来的。

But it gave us some evidence to go and say, we should try and build this thing.

但它给了我们一些证据，让我们可以说，我们应该尝试建造这个东西。

Itamar Gilad: So initially you fake it, mid-level tests are about building a rough version of it, it's not complete, it's not polished, it's not scalable, but it's good enough to give to users to start using.

Itamar Gilad：所以最初你假装它，中级测试是关于构建它的粗略版本，它不完整，它没有完善，它不可扩展，但它足以让用户开始使用。

So those are early adopter programs, alphas, longitudinal user studies and fish food.

这些是早期采用者计划、阿尔法、纵向用户研究和鱼食。

Fish food is testing on your own team.

鱼食正在您自己的团队中进行测试。

Lenny: Fish food?

莱尼：鱼食？

I haven't heard that term before.

我以前没听说过这个词。

So it's dog fooding, but more local to your team.

所以这是狗粮，但对你的团队来说更本地化。

Itamar Gilad: I think it's a Googly thing, but some people told me that they use fish food as well in their company the name.

Itamar Gilad：我认为这是 Google 的事情，但有些人告诉我，他们在公司名称中也使用鱼食。

So I'm using it, I don't know if there's a better name for it.

所以我正在使用它，我不知道是否有更好的名字。

Lenny: I wonder why it's called fish food, because it's like little?

莱尼：我想知道为什么它被称为鱼食，因为它看起来很小？

It's like little gentle little clicks?

就像轻微的轻微点击声？

Itamar Gilad: It could be.

伊塔玛·吉拉德：有可能。

Yeah, I don't know.

是的，我不知道。

Lenny: Wow.

莱尼：哇。

Okay, super cool.

好的，超级酷。

I'm learning a lot here.

我在这里学到了很多东西。

Itamar Gilad: So the next stage is to actually build a kind of more complete version of this and then you can dog food it, then you can give this to your users internally.

Itamar Gilad：所以下一阶段是实际构建一种更完整的版本，然后你可以对其进行测试，然后你可以在内部将其提供给你的用户。

When I joined Microsoft many years ago, the first thing I noticed was that Outlook was very buggy and I asked people what's going on?

多年前，当我加入 Microsoft 时，我注意到的第一件事是 Outlook 有很多 bug，我问人们这是怎么回事？

And they told me we are all dog fooding the next version of Outlook that hasn't come out yet and that's a very common practice in Silicon Valley.

他们告诉我，我们都在等待尚未发布的下一版本的 Outlook，这在硅谷是一种非常常见的做法。

You can do previews, you can do betas, you can do labs, so those are tests.

你可以做预览，你可以做测试版，你可以做实验室，所以这些都是测试。

Now, there's a special class of tests which are experiments because they have a control element.

现在，有一类特殊的测试是实验，因为它们有一个控制元素。

So AB tests, multivariate tests, those are all experiments.

所以AB测试、多变量测试，这些都是实验。

I'm using the word experiment the way data scientists use it, although people tend to call experiments to everything that you see here and finally, even the release you can do stage release, you can do percent launches, you can do hold backs.

我使用“实验”这个词的方式与数据科学家使用它的方式相同，尽管人们倾向于将你在这里看到的所有内容称为实验，最后，即使是发布，你可以进行阶段发布，你可以进行百分比发布，你可以进行保留。

All of these things help you further validate your assumptions.

所有这些都可以帮助您进一步验证您的假设。

Sometimes you need to roll back and change things, but it's another opportunity to learn.

有时你需要回滚并改变一些事情，但这是另一个学习的机会。

Itamar Gilad: So the key point is you don't have to start at the right-hand side, which is expensive.

Itamar Gilad：所以关键点是你不必从右侧开始，这很昂贵。

You can start early on and that leads to poking a lot of ideas very quickly.

你可以尽早开始，这样就能很快提出很多想法。

You realize they're not as good as you thought, and then you can invest more effort into the good ideas.

你意识到它们并不像你想象的那么好，然后你可以投入更多的精力来研究好想法。

If they generate positive evidence, you can go further and further until that point where you feel you're ready for delivery.

如果他们产生了积极的证据，你就可以走得越来越远，直到你觉得你已经准备好分娩了。

Lenny: Okay.

莱尼：好的。

So we've talked about goals, we've talked about ideas, we're talking about steps here.

所以我们讨论了目标，我们讨论了想法，我们在这里讨论了步骤。

Is there anything else along steps?

步骤还有其他吗？

And then next I know comes tasks.

接下来我知道任务来了。

Itamar Gilad: No, this is it for steps.

Itamar Gilad：不，这就是步骤。

There's a lot more with this, we will not go into all of it.

还有很多事情，我们不会全部讨论。

Lenny: Okay, that sounds good.

莱尼：好的，听起来不错。

Let's talk about tasks and what you mean there.

让我们谈谈任务以及您的意思。

Itamar Gilad: All right, awesome.

Itamar Gilad：好的，太棒了。

So in many organizations there's these two worlds.

所以在许多组织中都存在这两个世界。

There's the planning world where basically you have the managers, the stakeholders, some of the PMs really sit and think about what we need to launch and that's where we create the strategies and the roadmaps and the projects.

在规划世界中，基本上有经理、利益相关者、一些项目经理真正坐下来思考我们需要推出什么，这就是我们制定战略、路线图和项目的地方。

But guess who is not invited to the party?

但猜猜谁没有被邀请参加聚会？

The people who are actually doing the work.

真正在做这项工作的人。

They live in Agile world, they're very focused on moving tickets to the done state, on completing burning story points, pushing stuff into production and there's a big gap between these two worlds.

他们生活在敏捷世界中，他们非常专注于将门票转移到完成状态，完成燃烧的故事点，将东西投入生产，这两个世界之间存在很大的差距。

They don't understand each other, they don't see eye to eye, there's a lot of mistrust being built sometimes against the plans or the managers feels that the teams are just not being very effective.

他们彼此不理解，意见不一致，有时会对计划产生很多不信任，或者管理者认为团队效率不高。

We've seen all of this and the solution, the stop gap is to put a PM in the middle.

我们已经看到了所有这些和解决方案，权宜之计就是在中间放一个 PM。

The PM is supposed to make all of this work, deliver on the roadmap like a project manager, feed the Agile machine with perfectly prioritized product backlogs and stories and it just doesn't work honestly.

PM 应该完成所有这些工作，像项目经理一样交付路线图，为敏捷机器提供完全优先的产品待办事项和故事，但它只是不能诚实地工作。

Itamar Gilad: And the PMs I meet are very tired and they have to spend so much time in planifications and roadmap discussions and they're very busy, they don't have time to do research or to test ideas.

Itamar Gilad：我遇到的产品经理非常疲惫，他们必须花费大量时间进行规划和路线图讨论，而且他们非常忙碌，没有时间进行研究或测试想法。

So I suggest changing this and bringing the developers a little bit out of their Agile cage if you like and no disrespect to Agile, it's a great thing but let's let them do more than just develop.

因此，我建议改变这一点，并让开发人员稍微走出他们的敏捷笼子，如果你愿意并且没有对敏捷的不尊重，这是一件很棒的事情，但让我们让他们做的不仅仅是开发。

Let's let them discover as well and one of the tools I suggest and again this is a process is what I call the GIST board.

让我们也让他们发现我建议的工具之一，这也是一个过程，我称之为 GIST 板。

So it's basically the top three layers of GIST.

所以它基本上是胃肠道间质瘤的前三层。

The goals are on the right, these are just the key results usually per team I suggest not more than four.

目标在右侧，这些只是关键结果，通常我建议每个团队不要超过四个。

So you create a GIST board per team, then the ideas we're working on right now sometimes with our ICE scores and then the next few steps that we might want to pursue in order to validate these ideas and this is a very dynamic thing.

因此，你为每个团队创建一个 GIST 板，然后是我们现在正在研究的想法（有时会使用我们的 ICE 分数），然后是我们可能想要采取的接下来的几个步骤，以验证这些想法，这是一个非常动态的事情。

Itamar Gilad: It changes all the time, the team leads need to update it and the team needs to meet around it at least once every other week to think to talk about what's going on.

Itamar Gilad：它一直在变化，团队领导需要更新它，团队需要至少每隔一周召开一次会议，思考讨论正在发生的事情。

Are we still following the right ideas?

我们仍然遵循正确的想法吗？

How are we doing on the goals?

我们在目标上做得怎么样？

What are the next steps?

接下来的步骤是什么？

What's blocking us from completing the most important steps?

是什么阻碍我们完成最重要的步骤？

And this is a discussion that is not happening today, because most of the discussion happens at the roadmap level and then there's a lot of discussion at the task level.

这是今天不会发生的讨论，因为大多数讨论发生在路线图级别，然后在任务级别有很多讨论。

But this middle layer of what actually are we trying to achieve and how well are we doing on it doesn't exist.

但我们实际想要实现的目标以及我们在这方面做得如何的中间层并不存在。

If you do have this, you create a lot more context in the minds of your team and then they need to ask you fewer questions.

如果你确实做到了这一点，你就在团队的头脑中创造了更多的背景，然后他们需要问你更少的问题。

You need to tell them less what to do.

你需要更少地告诉他们该做什么。

They know what's success and they are able to actually do a lot more on their own.

他们知道什么是成功，而且他们实际上能够自己做更多的事情。

Lenny: Is the way to think about the GIST board as the way you should be road roadmapping or is this more of a strategy framework to think about why you should be prioritizing broadly?

Lenny：将 GIST 委员会视为您应该制定路线图的方式，还是这更像是一个战略框架，用于思考为什么您应该广泛优先考虑？

Itamar Gilad: The way I say this is at the beginning of the quarter, the team defines its goals.

Itamar Gilad：我所说的方式是在季度初，团队定义其目标。

The leads of the team define the goals, but they review it with the team, they review it with the managers, of course with the stakeholders.

团队的领导者定义了目标，但他们与团队一起审查它，他们与经理一起审查，当然也与利益相关者一起审查。

Everyone's in agreement, these are the maximum four key results and the one or two objectives you guys need to work on, teams cannot deliver on more than that.

每个人都同意，这些是最多的四个关键结果以及你们需要努力实现的一两个目标，团队无法实现更多。

You copy these key results into the GIST board, then you start looking at your idea bank or you start generating ideas and say, how can we achieve these key results?

你将这些关键结果复制到 GIST 板上，然后你开始查看你的想法库，或者你开始产生想法并说，我们如何才能实现这些关键结果？

Lenny: And to clarify the thing you copy is the key result as the goal?

Lenny：澄清你复制的东西是以关键结果为目标？

Itamar Gilad: Yes, exactly.

伊塔玛·吉拉德：是的，完全正确。

You can write the objectives alongside that to remind people what are we trying to achieve, but the key results are the thing we show here.

您可以将目标写在旁边，以提醒人们我们要实现什么目标，但关键结果是我们在这里展示的内容。

Then you pick some ideas, the ones that look most promising and as unintuitive as it sounds or counterintuitive as this sounds I would recommend that you let the team pick these ideas.

然后你选择一些想法，那些看起来最有前途的想法，虽然听起来不直观，但听起来却违反直觉，我建议你让团队选择这些想法。

The manager of the stakeholders can propose the ideas, everyone can propose, but the team should use the ICE process to kind of...

利益相关者的经理可以提出想法，每个人都可以提出，但团队应该使用 ICE 流程来......

And especially the product manager is very important here to choose which ideas to test first.

尤其是产品经理在这里选择首先测试哪些想法非常重要。

Then the team together needs to develop which steps should we run, how can we validate this?

然后团队需要共同制定我们应该运行哪些步骤，我们如何验证这一点？

Some of the steps will be done by the PM, some by the data analyst, some by the user researcher.

有些步骤将由产品经理完成，有些由数据分析师完成，有些由用户研究员完成。

But some will involve the team, there'll be some coding, there'll be some running of experiments and so there's some ownership around the steps.

但有些会涉及团队，会有一些编码，会有一些实验运行，所以围绕这些步骤有一些所有权。

A sub team owns each one of these steps and we will change the board very actively.

这些步骤中的每一个步骤都由一个子团队负责，我们将非常积极地更换董事会。

Itamar Gilad: So if an idea turns out to be bad we will take it off the board and put another idea in this place or maybe we achieve the goal, we don't need to work on this anymore, we can focus something else.

Itamar Gilad：因此，如果一个想法被证明是糟糕的，我们会将其从董事会中删除，并在这个地方放入另一个想法，或者也许我们实现了目标，我们不需要再为此工作，我们可以专注于其他事情。

So it's a project management tool in a sense.

所以从某种意义上来说它是一个项目管理工具。

Lenny: Awesome.

莱尼：太棒了。

So I'm looking at it and I think maybe the most important piece of this is that steps aren't just like a project, like launch a better onboarding or add the step to onboarding.

因此，我正在研究它，我认为其中最重要的部分可能是步骤不仅仅是一个项目，例如启动更好的入职或在入职中添加步骤。

It's you want to emphasize the steps that you're going to take to get to more and more confidence essentially, and more and more evidence guided thinking versus just, "Well, let's figure out how to launch this feature idea."

你想要强调的是，你将采取哪些步骤来获得越来越多的信心，以及越来越多的证据引导思维，而不是仅仅“好吧，让我们弄清楚如何推出这个功能想法”。

Itamar Gilad: Exactly.

伊塔玛·吉拉德：没错。

It's not a engineering milestone or a design milestone, it's a learning milestone.

这不是工程里程碑或设计里程碑，而是学习里程碑。

So we build something and along the way we actually grow the scope of what we build.

因此，我们构建了一些东西，并在此过程中我们实际上扩大了我们构建的范围。

We are building the product in the process and we learn, so the two have to come hand in hand.

我们在这个过程中构建产品并学习，因此两者必须齐头并进。

Lenny: And for folks that aren't watching this on YouTube, just to walk through an example, we'll do it real quick.

Lenny：对于那些没有在 YouTube 上观看此内容的人，我们会快速演示一个示例。

So one of your goals here is average onboarding time, you want your goal to be the average onboarding time less than two days, currently five and a half days.

因此，您的目标之一是平均入职时间，您希望您的目标是平均入职时间少于两天，目前为五天半。

An idea there is an onboarding wizard, and then the steps are a usability test with mockups and then a usability test as a prototype and then an AB test?

一个想法是有一个入门向导，然后步骤是使用模型进行可用性测试，然后作为原型进行可用性测试，然后进行 AB 测试？

Itamar Gilad: Yeah, basically, and you can alter this as you go along.

Itamar Gilad：是的，基本上，你可以在进行过程中改变这一点。

Sometimes you can run multiple steps in parallel it's not always sequential.

有时您可以并行运行多个步骤，但并不总是按顺序运行。

But that's basically the process, yeah.

但这基本上就是这个过程，是的。

Lenny: Awesome.

莱尼：太棒了。

So again, what you're trying to emphasize here as a team is just we're not just going to launch this onboarding wizard and we're not going to figure it out later.

再说一遍，作为一个团队，您在这里想强调的是，我们不仅仅是要启动这个入门向导，而且我们以后也不会弄清楚它。

It's like let's be upfront about the steps we're going to take to build more and more confidence.

这就像让我们坦率地说明我们将要采取的步骤来建立越来越多的信心。

This is something we should keep investing more and more in, which is really interesting.

这是我们应该继续加大投入的事情，这非常有趣。

Itamar Gilad: Yeah, and another interesting thing that happens every time you run a step if it's successful you have evidence and you can go back to the managers and tell them and share and say, "With this idea we thought it was great, but we got this result.

Itamar Gilad：是的，还有一件有趣的事情，每次你运行一个步骤时都会发生，如果它成功了，你就有证据，你可以回到经理那里告诉他们并分享并说，“有了这个想法，我们认为它很棒，但我们得到了这个结果。

What do you think that means?" And sometimes that manager that propose it would say, "I think the test failed, let's rerun it." Or sometimes they will say, "Maybe it's not as strong as I thought.

你认为这意味着什么？”有时，提出建议的经理会说：“我认为测试失败了，让我们重新运行它。”或者有时他们会说：“也许它没有我想象的那么强大。

The discussion just becomes that much more nuanced and objective if you like.

如果你愿意的话，讨论会变得更加细致和客观。

Lenny: Maybe just to close out this framework.

莱尼：也许只是为了结束这个框架。

How does this relate to a roadmap that they may have in a spreadsheet or in Jira or in Asana or something like that.

这与他们在电子表格、Jira 或 Asana 或类似内容中可能拥有的路线图有什么关系。

Does this sit on top of that?

这是在上面吗？

Is this replacing a roadmap somewhere else?

这会取代其他地方的路线图吗？

Itamar Gilad: I would say that release roadmaps where you are just saying by Q3 we want to launch this or by October we have to launch that, they're kind of competing with this.

Itamar Gilad：我想说的是发布路线图，你只是说到第三季度我们要推出这个，或者到 10 月份我们必须推出那个，他们有点与这个竞争。

If you're doing that and people know that the goal is to launch that thing by October, forget about learning, forget about evidence guided, I recommend using outcome roadmaps saying by October we want to achieve this outcome.

如果你正在这样做，并且人们知道目标是在 10 月之前推出该产品，那么忘记学习，忘记证据引导，我建议使用结果路线图，说明我们希望在 10 月之前实现这一结果。

By Q4 we want to launch in another three countries, or we want to grow our usage in India by that much, by this time we need to tackle the problem of churn and how we achieve this.

到第四季度，我们希望在另外三个国家推出，或者我们希望将印度的使用量增加那么多，此时我们需要解决客户流失问题以及如何实现这一目标。

Sometimes we know we have a concrete idea that is high confidence that we already tested, we switch into delivery, then we can put it on the roadmap and say, "Yeah, we're going to build this thing and we'll aim for October." But otherwise you want to keep it open and the roadmaps can kind of suffocate this process if you decide upfront with low confidence that this particular idea must be launched.

有时我们知道我们有一个具体的想法，我们已经测试过，我们切换到交付，然后我们可以将其放在路线图上并说：“是的，我们将构建这个东西，我们的目标是十月份。”但除此之外，如果您希望保持开放性，并且如果您事先没有信心地决定必须启动这个特定的想法，那么路线图可能会窒息这个过程。

Lenny: Okay.

莱尼：好的。

So you're proposing people switch the roadmapping practice to this, which is very ambitious.

所以你建议人们将路线图实践转向这个，这是非常雄心勃勃的。

I love it.

我喜欢它。

Itamar Gilad: Well, this is not a roadmap.

Itamar Gilad：嗯，这不是路线图。

This is just a tool for the team to manage the project, but I have a proposal for outcome roadmaps inside the book.

这只是团队管理项目的一个工具，但我在书中有一个关于结果路线图的建议。

Lenny: Okay, awesome.

莱尼：好的，太棒了。

Okay.

好的。

So I was going to ask if people wanted to try this approach, the book is the best way to fully understand the framework and how implement it.

所以我想问人们是否想尝试这种方法，这本书是完全理解该框架以及如何实现它的最佳方式。

Itamar Gilad: That's one way.

Itamar Gilad：这是一种方式。

I have articles, I have resources on my site, but I try to condense much of what we just discussed in a lot more nuance in the book.

我有文章，我的网站上有资源，但我试图在书中以更细微的差别来浓缩我们刚刚讨论的大部分内容。

So if you are interested in that, I would give it a go.

所以如果你对此感兴趣，我会尝试一下。

Lenny: Awesome.

莱尼：太棒了。

Maybe just on the topic of OKRs real quick.

也许只是很快就 OKR 的话题。

How do OKRs connect to all this?

OKR 如何与这一切联系起来？

It sounds like broadly you kind of assume people will keep working on here's our metric or key results or objectives and then that plugs into this kind of GIST framework.

听起来，您大致上假设人们会继续致力于我们的指标或关键结果或目标，然后将其插入这种 GIST 框架中。

Itamar Gilad: So the metrics trees, plus your mission, plus the individual missions of the teams give you most of what you need to populate your OKRs.

Itamar Gilad：因此，指标树、您的任务以及团队的单独任务为您提供了填充 OKR 所需的大部分内容。

There's of course a process of alignment, top down, bottom up, side to side, which I talk a little bit about as well.

当然有一个对齐的过程，从上到下，从下到上，从一边到另一边，我也谈到了一些。

OKRs is a very rich topic, but those things are usually the core.

OKR 是一个非常丰富的主题，但这些东西通常是核心。

There's usually some other OKRs that's about the health of the company, the health of the product, etc. Those are called supplementary OKRs, I talk about those as well.

通常还有一些其他 OKR 与公司的健康状况、产品的健康状况等有关。这些称为补充 OKR，我也会谈论这些。

So yeah, I think OKRs are a helpful tool if you like them.

所以，是的，我认为 OKR 是一个有用的工具，如果你喜欢的话。

Lenny: And just zooming out again.

莱尼：然后再次缩小。

Basically you don't need to take all of these ideas and lump them all together and change the way you work as a business.

基本上，您不需要将所有这些想法整合在一起并改变您作为企业的工作方式。

You can start with picking some of these ideas and starting to become more and more evidence guided.

您可以从选择其中一些想法开始，并开始越来越多地以证据为指导。

It sounds like this GIST board isn't where you probably want to start, but maybe it's once you have more and more experience using some of these tools or you tell me.

听起来这个 GIST 板可能不是您想要开始的地方，但也许您在使用其中一些工具有了越来越多的经验或者告诉我之后就可以开始了。

Do you sometimes go straight to this way of thinking about the roadmap and the plan?

您有时会直接采用这种方式思考路线图和计划吗？

Itamar Gilad: So it might not be the full board because you're missing some of the pieces, maybe your goals are not as good or your idea prioritization isn't as good.

Itamar Gilad：所以这可能不是完整的董事会，因为你遗漏了一些部分，也许你的目标不够好，或者你的想法优先级不够好。

But if your team is very, very delivery focused and sometimes it's also the opposite.

但如果你的团队非常非常注重交付，有时情况恰恰相反。

The managers are telling them how to build and you want to break this kind of dynamic, you want to create a step backlog.

经理们告诉他们如何构建，而你想要打破这种动态，你想要创建一个步骤积压。

So instead of a product backlog, let's create a backlog of steps which are just validation steps, betas and previews, etc, and that changes the dynamic pretty strongly.

因此，让我们创建一个积压的步骤，而不是产品积压，这些步骤只是验证步骤、测试版和预览等，这会极大地改变动态。

Lenny: So by the time this podcast comes out, the book will be out.

莱尼：所以当这个播客出来时，这本书就会出版了。

What is the best place to find the book?

找到这本书的最佳地点是哪里？

Itamar Gilad: Hopefully on Amazon, you can search for it.

Itamar Gilad：希望您可以在亚马逊上搜索它。

You can go to my site, itamargilad.com and it'll be presented prominently there and there's also the book landings page where you'll find everything you need to know about the book, evidenceguided.com.

您可以访问我的网站 itamargilad.com，该网站将在该网站的显着位置显示，此外还有图书登陆页面，您可以在evidenceguided.com 上找到有关该书的所有信息。

Lenny: Well, with that we've reached our very exciting lightning round.

莱尼：好了，我们已经进入了非常激动人心的闪电回合。

Are you ready?

你准备好了吗？

Lenny: What are two or three books you've recommended most to other people?

莱尼：你向其他人推荐最多的两三本书是什么？

Itamar Gilad: So I'm going to cheat, I'm going to recommend a series of books so two series.

Itamar Gilad：所以我要作弊，我要推荐一系列书籍，所以两个系列。

One is the-

其一是——

Itamar Gilad: All right, cool.

Itamar Gilad：好吧，酷。

One, and those are obvious one.

一，这是显而易见的。

One is the series published by SVPG, Silicon Valley Product Group.

一个是SVPG（硅谷产品组）出版的系列。

So INSPIRED, EMPOWERED, now I think TRANSFORMED has come out, I haven't read it yet but I'm sure it's amazing.

所以受到启发，被赋予力量，现在我认为《转变》已经出版了，我还没有读过，但我确信它是惊人的。

So this is Marty Cagan and his colleagues, they write some tremendous books and every product manager should read them.

这就是马蒂·凯根和他的同事，他们写了一些很棒的书，每个产品经理都应该阅读它们。

The other series, a bit older, this is the Lean series, The Lean Startup, Lean Enterprise, Lean Analytics, there's gold in all these books, Lean UX, really, really important books and I think they're not as appreciated as they should.

另一个系列，有点老了，这是精益系列，精益创业，精益企业，精益分析，所有这些书中都有黄金，精益用户体验，非常非常重要的书籍，我认为它们没有得到应有的重视。

Running Lean, that's another example.

精益跑步，这是另一个例子。

Lenny: What is a favorite recent movie or TV show?

莱尼：最近最喜欢的电影或电视节目是什么？

Itamar Gilad: I'm not really a big TV or movie buff, I just put on whatever comes up.

Itamar Gilad：我并不是一个真正的电视或电影迷，我只是想看什么就看什么。

I'm discovering that YouTube is actually becoming one of my sources of information entertainment.

我发现 YouTube 实际上正在成为我的信息娱乐来源之一。

I'm learning a lot of Spanish recently, so I discovered this channel called Dreaming Spanish which is if you're learning Spanish it's incredible.

我最近学习了很多西班牙语，所以我发现了这个名为“梦想西班牙语”的频道，如果您正在学习西班牙语，那真是太棒了。

So that's my recommendation.

这就是我的建议。

Lenny: That's a unique choice, I love it.

莱尼：这是一个独特的选择，我喜欢它。

Favorite interview question you like to ask candidates.

您最喜欢问候选人的面试问题。

Itamar Gilad: I like to ask them to design something for a niche audience.

Itamar Gilad：我喜欢要求他们为小众受众设计一些东西。

So a navigation system for elderly people or some sort of laptop for people with vision impairment, etc. So those are good questions to see their customer empathy, their creativity, their ability to evaluate multiple ideas, their ability to find flaws in their own ideas.

因此，为老年人提供导航系统，或者为视力障碍者提供某种笔记本电脑等。这些都是很好的问题，可以了解他们的客户同理心、创造力、评估多种想法的能力、发现自己想法缺陷的能力。

So there's a lot of room to dig in there and kind of see how this person is thinking as a product person.

因此，这里有很大的空间可以挖掘，看看这个人作为产品人员是如何思考的。

Lenny: What is a favorite product you recently discovered that you love?

Lenny：您最近发现最喜欢的产品是什么？

Itamar Gilad: It's a cliche, but it's AI.

Itamar Gilad：这是陈词滥调，但它是人工智能。

There's a company called ElevenLabs, that do voices and the best voices, synthetic voices you heard, but they can also replicate your own voice so you can create a voice signature.

有一家名为 ElevenLabs 的公司，专门制作声音和您听到的最佳声音、合成声音，但他们也可以复制您自己的声音，以便您可以创建语音签名。

If you're American you can use their kind of default free version or cheap version to replicate your own voice and that could be pretty useful if you need to narrate an audiobook or do some online course.

如果您是美国人，您可以使用他们的默认免费版本或廉价版本来复制您自己的声音，如果您需要为有声读物进行旁白或进行一些在线课程，这可能非常有用。

So I'm finding this service very interesting.

所以我发现这项服务非常有趣。

Lenny: This is all part of my big retirement plan, find all of these components together that can replace me eventually.

莱尼：这是我伟大的退休计划的一部分，找到所有这些组件最终可以取代我。

You got AI generating content, we'll have this voice thing.

你有人工智能生成内容，我们将有这个语音东西。

I love it, it's all happening.

我喜欢它，这一切都在发生。

Itamar Gilad: There's an AI version of you, right?

Itamar Gilad：你有一个人工智能版本，对吧？

I can ask you questions now with-

我现在可以问你问题-

Lenny: It's all part of the plan.

莱尼：这都是计划的一部分。

Okay.

好的。

What is a favorite life motto that you repeat most to yourself that you share with others?

您最喜欢、最常对自己重复并与他人分享的人生格言是什么？

Itamar Gilad: That's a big one.

Itamar Gilad：这是一件大事。

Albert Einstein I think said, "Strive not to be a success, but to be of value." And I think that's a great motto for people and for companies.

我想阿尔伯特·爱因斯坦说过：“努力不是为了成功，而是为了有价值。”我认为这对于个人和公司来说都是一个伟大的座右铭。

It's something that kind of guides me and this whole concept of the value exchange, etc, is kind of loosely connected to that.

这是某种指导我的东西，价值交换的整个概念等等，与此有某种松散的联系。

Lenny: I love that, that's such a important point for people putting out content online.

莱尼：我喜欢这一点，这对于人们在网上发布内容来说非常重要。

So many people are just like, I just want to be successful, get followers, here's all these things I'm tweeting and showing and the thing that actually works is deliver value, create valuable stuff that people really value and want.

很多人都说，我只是想成功，获得追随者，这就是我在推特上发布和展示的所有这些东西，而真正有效的是传递价值，创造人们真正重视和想要的有价值的东西。

I find the signal for that is, do you find it interesting and valuable?

我发现的信号是，你觉得它有趣且有价值吗？

If you're like, "Oh wow, that's really interesting." Oftentimes other people are going to find it interesting.

如果你觉得，“哇哦，这真的很有趣。”通常其他人会发现它很有趣。

So I love that, great choice, I'm going to look at that one up.

所以我喜欢这个，很棒的选择，我会看看这个。

Two more questions.

还有两个问题。

What's the most valuable lesson you learned from your mom or your dad?

你从你的妈妈或爸爸那里学到的最有价值的教训是什么？

Itamar Gilad: I think both of them in their own way, they had relatively modest jobs, teaching or doing other things, but they always strived again to be the best they can and to deliver the most value they can.

伊塔玛·吉拉德：我认为他们两人都有自己的方式，他们的工作相对温和，教书或做其他事情，但他们总是再次努力做到最好，并提供最大的价值。

So it's very connected somehow, maybe I'm seeing the world through this lens.

所以它在某种程度上是紧密相连的，也许我是通过这个镜头看世界的。

But they kind of taught me to strive to be the best I can at what I do.

但他们教会了我在所做的事情上努力做到最好。

Lenny: The final question, you're Israeli for folks that can't tell.

莱尼：最后一个问题，对于那些无法分辨的人来说，你是以色列人。

What is your favorite Israeli food that people should definitely check out or I try to get whenever they can?

你最喜欢的以色列食物是什么，人们绝对应该去看看，或者我尽可能尝试去吃？

Itamar Gilad: When I arrive in Israel I usually go for shawarma, which is like döner kebab if you know it, it's just better.

Itamar Gilad：当我到达以色列时，我通常会吃沙威玛，如果你知道的话，这就像烤肉串，它只是更好。

So if you're in Israel, if you go visit Haifa, which is the city where I grew up definitely check out the shawarma.

因此，如果您在以色列，如果您去海法（我长大的城市）参观，一定要尝尝沙威玛。

Lenny: Awesome.

莱尼：太棒了。

Itamar, I hope people got the gist of your book from our conversation.

伊塔玛，我希望人们从我们的谈话中了解了你的书的要点。

What's the best way to find it?

找到它的最佳方法是什么？

What's the best way to learn about you and reach out if they want to ask any questions?

如果他们想问任何问题，了解您并联系您的最佳方式是什么？

And then also, how can listeners be useful to you?

还有，听众如何对你有用？

Itamar Gilad: To find it you can go to itamargilad.com or to evidenceguided.com, and you'll find a book and you'll find me.

Itamar Gilad：要找到它，您可以访问 itamargilad.com 或evidenceguided.com，您会找到一本书，也会找到我。

Best value to me, try it out, just take some of these ideas, bring them back to your office, talk with your colleagues, say what do you think we should do about this?

对我来说最有价值的是，尝试一下，只需将其中一些想法带回您的办公室，与您的同事交谈，说您认为我们应该对此做什么？

Just give it a go and reach back to me, tell me I'm easy to find in my website.

只要尝试一下并回复我，告诉我在我的网站上很容易找到我。

Tell me what happened I'm really interested.

告诉我发生了什么，我真的很感兴趣。

Lenny: Amazing.

莱尼：太棒了。

Itamar, thank you again so much for being here.

Itamar，再次非常感谢您来到这里。

Lenny: Bye everyone.

莱尼：大家再见。

Thank you so much for listening.

非常感谢您的聆听。

If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify or your favorite podcast app.

如果您发现这很有价值，您可以在 Apple Podcasts、Spotify 或您最喜欢的播客应用程序上订阅该节目。

Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast.

另外，请考虑给我们评分或留下评论，因为这确实有助于其他听众找到播客。

You can find all past episodes or learn more about the show at lennyspodcast.com.

您可以在 lennyspodcast.com 上找到所有过去的剧集或了解有关该节目的更多信息。

See you in the next episode.

下一集见。