How to scrappily hire for, measure, and unlock growth | Crystal Widjaja, Gojek and Kumu

Crystal Widjaja: I felt like it was a problem that was very solvable.

Crystal Widjaja：我觉得这是一个非常可以解决的问题。

And we ended up renting a stadium to just hire 60,000 drivers in a couple of weeks.

我们最终租了一个体育场，在几周内雇佣了 60,000 名司机。

So I think looking back, it was certainly a risk.

所以我认为现在回想起来，这确实是一个风险。

When I got there it was in a house and I realized I've probably made a huge mistake, but we were growing very quickly already, even at that small scale of 4,000 orders per day.

当我到达那里时，它在一所房子里，我意识到我可能犯了一个巨大的错误，但我们已经发展得很快，即使是每天 4,000 个订单的小规模。

Lenny: Crystal Widjaja has been leading product and growth teams at some of the largest consumer businesses in Southeast Asia, including Kumu, where she's currently the chief product officer, and Gojek where she built and led the growth team through the early years of what is now the largest super app in Southeast Asia.

Lenny：Crystal Widjaja 一直在东南亚一些最大的消费企业领导产品和增长团队，其中包括 Kumu（她目前担任该企业的首席产品官）和 Gojek（她在 Gojek 中组建并领导了增长团队），该团队现已成为东南亚最大的超级应用程序。

To put this in context, Gojek completes more rights per day than Lyft and more food deliveries than GrubHub, Uber Eats and DoorDash combined, and it's the number one mobile wallet in Indonesia and Southeast Asia.

综合来看，Gojek 每天完成的权利比 Lyft 还多，送餐数量比 GrubHub、Uber Eats 和 DoorDash 的总和还要多，而且它是印度尼西亚和东南亚排名第一的移动钱包。

In my opinion, American startups have a lot to learn from startups in Asia and Crystal has been at the ground floor of some of the biggest successes there.

在我看来，美国初创企业有很多值得向亚洲初创企业学习的地方，而 Crystal 一直是亚洲一些最成功企业的基础。

Lenny: In our conversation, we covered the biggest growth unlocks that Crystal has seen across the companies she's worked at, what growth investments usually pay off and which often don't, we dig into growth models, a bunch of tips for accelerating growth, why most analytics efforts fail in how to avoid that, how to hire and structure your growth team, and we also talk about the nonprofit that Crystal started that aims to help young women get into STEM called Generation Girl.

Lenny：在我们的谈话中，我们讨论了 Crystal 在她工作过的公司中看到的最大的增长解锁，哪些增长投资通常会带来回报，哪些通常不会带来回报，我们深入研究了增长模型，一系列加速增长的技巧，为什么大多数分析工作在如何避免这种情况方面失败了，如何雇用和组建你的增长团队，我们还讨论了 Crystal 创办的非营利组织，旨在帮助年轻女性进入 STEM，称为 Generation Girl。

Crystal is such a star, and I hope that you enjoy this episode as much as I did.

克里斯托是一位明星，我希望你和我一样喜欢这一集。

And with that, I bring you Crystal Widjaja.

接下来，我为您带来水晶 Widjaja。

If you're setting up your analytics stack, but you're not using Amplitude, what are you doing?

如果您正在设置分析堆栈，但没有使用 Amplitude，那么您在做什么？

Amplitude is the number one most popular analytic solution in the world used by both big companies like Shopify, Instacart, and Atlassian, and also most tech startups.

Amplitude 是世界上最受欢迎的分析解决方案，Shopify、Instacart 和 Atlassian 等大公司以及大多数科技初创公司都在使用。

Lenny: Amplitude has everything you need, including a powerful and fully self-service analytics product, an experimentation platform, and even an integrated customer data platform to help you understand your users like never before.

Lenny：Amplitude 拥有您所需的一切，包括功能强大且完全自助的分析产品、实验平台，甚至是集成的客户数据平台，可帮助您以前所未有的方式了解用户。

Give your teams self-service product data to understand your users, drive conversions and increase engagement, growth, and revenue.

为您的团队提供自助产品数据，以了解您的用户、推动转化并提高参与度、增长和收入。

Get your vanity metrics, trust your data, work smarter and grow your business.

获取您的虚荣指标，信任您的数据，更智能地工作并发展您的业务。

Try Amplitude for free.

免费试用 Amplitude。

Just visit amplitude.com to get started.

只需访问振幅.com 即可开始。

Hey, Ashley, head of marketing at Flatfile.

嘿，Ashley，Flatfile 的营销主管。

How many B2B SaaS companies would you estimate need to import CSV files from their customers?

您估计有多少家 B2B SaaS 公司需要从其客户导入 CSV 文件？

Lenny: And how many of them screw that up, and what happens when they do?

莱尼：有多少人把事情搞砸了，当他们这样做时会发生什么？

Ashley: Well, based on our data about a third of people will consider switching to another company after just one bad experience during onboarding.

阿什利：嗯，根据我们的数据，大约三分之一的人在入职期间经历过一次糟糕的经历后就会考虑跳槽到另一家公司。

So if your CSV importer doesn't work right, which is super common, considering customer files are chalked full of unexpected data and formatting, they'll leave.

因此，如果您的 CSV 导入器无法正常工作（这种情况非常常见），考虑到客户文件中充满了意外的数据和格式，他们就会离开。

Lenny: I am zero percent surprised to hear that.

莱尼：听到这个消息我感到百分之零的惊讶。

I've consistently seen that improving onboarding is one of the highest leverage opportunities for both signup conversion and increasing longterm retention, getting people to your aha moment more quickly and reliably is so incredibly important.

我一直认为，改善入职是注册转化和提高长期保留率的最高杠杆机会之一，让人们更快、更可靠地进入顿悟时刻非常重要。

Ashley: Totally.

阿什利：完全可以。

It's incredible to see how our customers like Square, Spotify and Zora are able to grow their businesses on top of Flatfile.

看到 Square、Spotify 和 Zora 等客户如何能够在 Flatfile 上发展业务，真是令人难以置信。

It's because flawless data onboarding acts like a catalyst to get them and their customers where they need to go, faster.

这是因为完美的数据导入就像催化剂一样，可以让他们及其客户更快地到达他们需要去的地方。

Lenny: If you'd like to learn more or get started, check out Flatfile at flatfile.com/lenny.

Lenny：如果您想了解更多信息或开始使用，请访问 flatfile.com/lenny 查看 Flatfile。

Crystal, thank you so much for being here.

水晶，非常感谢你来到这里。

I've read a bunch of your stuff online.

我在网上读过你的很多东西。

We've exchanged a bunch of emails and tweets, but this is the first time that we're actually chatting for real, and so I'm really excited to, yeah I'm excited to learn from you and for folks to learn about you.

我们交换了很多电子邮件和推文，但这是我们第一次真正地聊天，所以我真的很兴奋，是的，我很高兴向你学习，也很高兴人们了解你。

Crystal Widjaja: We may have crossed paths on Clubhouse and the audio forums or on the Twitterverse.

Crystal Widjaja：我们可能在 Clubhouse、音频论坛或 Twitterverse 上有过交集。

So it's really cool to see you.

所以很高兴见到你。

Lenny: Wow.

莱尼：哇。

I just remembered that.

我刚刚想起来了。

That is so right.

说得太对了。

I think we were talking about Reforge and Eppo.

我想我们正在谈论 Reforge 和 Eppo。

Is that right?

是这样吗？

Crystal Widjaja: Yes, that's right.

Crystal Widjaja：是的，没错。

Good times.

美好时光。

Crystal Widjaja: The days when club does was a thing.

Crystal Widjaja：俱乐部的日子很重要。

They have a thing too, to learn about exponential decay.

他们还有一件事，就是了解指数衰减。

Lenny: Oh man.

莱尼：天啊。

Okay.

好的。

Maybe we'll get to that.

也许我们会做到这一点。

And we're going to be chatting a lot about consumer growth and a bunch of stuff along those lines.

我们将讨论很多关于消费者增长和类似问题的内容。

But before you get into that, you have a fairly unusual path and also geography as compared to many of my other guests.

但在你开始之前，与我的许多其他客人相比，你有一条相当不寻常的道路和地理位置。

And so just to set a little context, could you just walk us through your career path and journey from, I think as an investment banker initially, and then currently as chief product officer at Kumu and then living in Singapore also?

因此，为了介绍一些背景，您能否向我们介绍一下您的职业道路和旅程，我认为最初是一名投资银行家，然后目前担任 Kumu 的首席产品官，然后也住在新加坡？

So yeah, tell us all about your path.

所以，是的，告诉我们你的道路。

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

I think my path was certainly a nonstandard one.

我想我的路肯定是一条不规范的路。

While I grew up in San Jose, the Bay Area, you could see companies like Lyft emerging around the year that I was graduating college.

我在湾区圣何塞长大，在我大学毕业的那一年，你可以看到像 Lyft 这样的公司涌现。

But really it was how do I graduate college as quickly as possible because this is very boring.

但实际上这是我如何尽快从大学毕业，因为这很无聊。

So I took a Poli Sci major.

所以我选了政治科学专业。

I am not a math or a computer science major.

我不是数学或计算机科学专业的。

I didn't know what a consultant was because I was just trying to get out of college.

我不知道什么是顾问，因为我只是想从大学毕业。

I didn't realize people start looking for jobs before they'd graduate.

我没有意识到人们在毕业之前就开始找工作。

So the last two weeks of school I was looking on Craigslist, because I was like, "Craigslist is how everyone gets a job." I'm a first generation American student, so my parents could not help me at all with like, "You should look into this company called McKinsey," or, "Here are all of the life paths that you have ahead of you." So I ended up taking an investment banking research job.

所以在学校的最后两周，我在 Craigslist 上查找，因为我想，“Craigslist 是每个人找到工作的方式。”我是第一代美国学生，所以我的父母根本无法帮助我，比如“你应该看看这家叫麦肯锡的公司”，或者“这是你面前的所有人生道路。”所以我最终接受了一份投资银行研究工作。

Crystal Widjaja: And my job there was to figure out how to call startups and analyze their potential for VC financing or M&A advisory.

Crystal Widjaja：我在那里的工作是弄清楚如何给初创公司打电话并分析它们在风险投资或并购咨询方面的潜力。

And I barely knew what those words meant at the time.

当时我几乎不知道这些话的含义。

I ended up owning a huge Excel database of 130,000 rows, 60 plus columns.

我最终拥有了一个包含 130,000 行、60 多列的庞大 Excel 数据库。

And because again, I am very impatient, I was like, "This is a terrible experience.

因为我再次非常不耐烦，我想，“这是一次可怕的经历。

How would I create a customer database?" And so I ended up Google fooling all of the work needed to build a MySQL database, I presented a plan, and investment banking, surprise, surprise, is not very tech forward.

我如何创建客户数据库？”因此，我最终让 Google 愚弄了构建 MySQL 数据库所需的所有工作，我提出了一个计划，而投资银行业务，令人惊讶的是，技术并不是很先进。

So they looked at my plans and they're like, "What is this MySQL thing?

所以他们看了我的计划然后说：“MySQL 是什么？

Isn't that super expensive?

是不是超级贵啊？

What is open source?" So I ended up leaving that job because I realized that if I wanted to get into something more tech, it would probably not be at a investment bank.

什么是开源？”所以我最终离开了那份工作，因为我意识到，如果我想进入科技含量更高的领域，可能不会去投资银行。

So I did took the investment banking strategies that I had learned there and applied the same pattern matching to companies in Southeast Asia.

因此，我确实采用了在那里学到的投资银行策略，并将相同的模式应用于东南亚的公司。

Crystal Widjaja: So my family originally is from Indonesia.

Crystal Widjaja：所以我的家人来自印度尼西亚。

I thought I have a kind of safety net, I must speak Indonesian really well just by birth, so maybe that's a great country for me to look at.

我以为我有一个安全网，我一出生就必须说得很好，所以也许那对我来说是一个很棒的国家。

So I took the approach of let's find a company that makes a lot of sense, that I feel I resonate with.

所以我采取的方法是，让我们找到一家很有意义、我觉得能引起共鸣的公司。

And I literally cold called emails some companies.

我真的给一些公司发了电子邮件。

So Gojek being on that list.

所以 Gojek 就在这个名单上。

I literally emailed someone after Googling HR at Gojek and said, "I'm willing to move to Indonesia, take a bet on me." And they actually did.

在谷歌上搜索 Gojek 的人力资源后，我确实给某人发了一封电子邮件，说：“我愿意搬到印度尼西亚，你可以在我身上打赌。”他们确实做到了。

So I got extremely lucky.

所以我非常幸运。

Five years fly by insanely fast.

五年的时间过得真快。

I went through building out the data team from scratch.

我从头开始组建了数据团队。

When you have all of the data, you know how much fraud you have in the system.

当你掌握了所有数据后，你就知道系统中有多少欺诈行为。

So then I ended up building out the fraud and risk team, picked up performance marketing, and then it was like, okay, now we're ready to grow.

因此，我最终建立了欺诈和风险团队，选择了绩效营销，然后就好像，好吧，现在我们已经准备好成长了。

So you have all of this data now take on growth.

现在所有这些数据都在增长。

Lenny: Got it.

莱尼：明白了。

You were very modest about Gojek and the success of that company, and also Kumu where you work now.

您对 Gojek 和该公司的成功以及您现在工作的 Kumu 非常谦虚。

So just to set a little context for folks that aren't familiar with these companies, can you share how big they are and how big of a deal they are in Southeast Asia?

那么，为了给那些不熟悉这些公司的人们提供一些背景信息，您能否分享一下它们的规模以及它们在东南亚的影响力有多大？

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

They are pretty massive.

它们非常庞大。

So Gojek is now called GoTo, they just merged with the largest eCommerce platform in Indonesia.

所以 Gojek 现在更名为 GoTo，他们刚刚与印度尼西亚最大的电子商务平台合并。

So across Southeast Asia, we had about 170 million users.

因此，在整个东南亚，我们拥有大约 1.7 亿用户。

Southeast Asia has scale.

东南亚有规模。

If you ever wanted to work at scale, you would go to Southeast Asia.

如果你想大规模工作，你会去东南亚。

We had 20 plus different services from transportation to food, shopping, medicine delivery, bill pay, movie tickets.

我们提供 20 多种不同的服务，从交通到食品、购物、药品递送、账单支付、电影票。

So it was like all of the startups in America in one app, all being built at the same time with the same user base.

所以这就像美国所有的初创公司都在一个应用程序中一样，都是在相同的用户基础上同时构建的。

And so everything was tremendously layered, because you could fill all of these opportunity gaps in the market where a single app would probably not be as sustainable.

因此，一切都是非常分层的，因为你可以填补市场上所有这些机会空白，而单个应用程序可能不那么可持续。

Crystal Widjaja: So Gojek is massive across Indonesia, Singapore, Thailand, Vietnam.

Crystal Widjaja：Gojek 在印度尼西亚、新加坡、泰国和越南都很受欢迎。

And then Kumu is kind of a super app for social.

Kumu 是一款超级社交应用程序。

So Gojek was very transactional.

所以 Gojek 非常善于交易。

It was like, "Here's a job to be done.

就像是，“这里有一项工作要做。

I want to pay for something and someone delivers it to me." And with Kumu, it's more so of a, "I want to do clubhouse, Zoom, Google Hangouts gather around all in one app." So we cover social feeds, audio, video, multi seats.

我想付钱买某样东西，有人把它送给我。”对于 Kumu，更重要的是，“我想将俱乐部会所、Zoom、Google Hangouts 全部集中在一个应用程序中。”因此，我们涵盖社交信息流、音频、视频、多席位。

There's a ton of different use cases that we serve on Kumu.

我们在 Kumu 上提供了大量不同的用例。

And Kumu is primarily in the Philippines, but ranks top 10 and a bunch of countries as a top grossing mobile app.

Kumu 主要在菲律宾，但在最畅销的移动应用程序中位列前 10 名和许多国家。

Lenny: So with Kumu you joined when they're already doing fairly well.

Lenny：所以在 Kumu 已经做得相当不错的时候你就加入了。

But Gojek, as you said, you joined very early.

但是Gojek，正如你所说，你很早就加入了。

What did you see in that company that helped you decide to join such a risky, early stage company?

您在那家公司看到了什么，让您决定加入这样一家风险很大的早期公司？

For folks that are maybe thinking about joining a startup, what kind of things did you take away of what to look for?

对于那些可能正在考虑加入一家初创公司的人来说，你从寻找的东西中拿走了哪些东西？

Crystal Widjaja: Honestly, it's probably a lot of luck.

Crystal Widjaja：说实话，这可能需要很大的运气。

But also at that age I realized I have very little to lose.

但也在那个年纪我意识到我已经没什么可失去的了。

So with Gojek I think I felt like it was the right company because I was able to really clearly understand the value prop.

因此，我认为 Gojek 是一家合适的公司，因为我能够真正清楚地理解其价值支柱。

Traffic in Indonesia is crazy.

印度尼西亚的交通很疯狂。

It takes you two hours to go 20 kilometers.

20公里需要两个小时。

So of course you want to take a motorcycle taxi to beat that traffic.

因此，您当然想乘坐摩托车出租车来避开交通拥堵。

Of course, you don't want to go out and get food and then have to come back this long pathway of two hours.

当然，你不想出去买食物，然后又要经过两个小时的漫长路程回来。

So I think taking that Warren Buffet approach, I knew that the product made sense.

所以我认为采用沃伦·巴菲特的方法，我知道这个产品是有意义的。

The market made sense as well.

市场也是有道理的。

So drivers, there were already a thing, but it was very hard to connect them to the consumer.

所以司机们已经有了一个东西，但很难将它们与消费者联系起来。

Crystal Widjaja: It was painful to haggle prices.

Crystal Widjaja：讨价还价很痛苦。

There were lots of restaurants scattered across Indonesia.

印度尼西亚各地分布着许多餐馆。

So the value prop and the market made sense and the channel by which you would do it through this mobile app made a little bit less sense at the time because most drivers didn't have a mobile app, but I felt like it was a problem that was very solvable.

因此，价值支柱和市场是有意义的，而通过这个移动应用程序来实现这一目标的渠道当时意义不大，因为大多数司机没有移动应用程序，但我觉得这是一个非常可以解决的问题。

And we ended up renting a stadium to just hire 60,000 drivers in a couple of weeks.

我们最终租了一个体育场，在几周内雇佣了 60,000 名司机。

So I think looking back, it was certainly a risk when I got there, it was in a house and I realized I've probably made a huge mistake.

所以我想回想起来，当我到达那里时，这确实是一个风险，那是在一所房子里，我意识到我可能犯了一个巨大的错误。

But we were growing very quickly already, even at that small scale of 4,000 orders per day.

但我们的增长速度已经非常快了，即使是每天 4,000 个订单的小规模也是如此。

Lenny: I want to spend a lot of time talking about what you learned, driving growth at these companies.

莱尼：我想花很多时间谈论你学到的东西，推动这些公司的增长。

But one quick question.

但有一个简单的问题。

So, Gojek's the super app where you do a lot of stuff in one app.

所以，Gojek 是一款超级应用程序，您可以在一个应用程序中完成很多操作。

Do you have any insights into why a super app hasn't emerged in the US?

您对为什么美国没有出现超级应用程序有什么见解吗？

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

I think the sentimentality of a conglomerate is very different in Southeast Asia.

我认为东南亚企业集团的情感是非常不同的。

So we've grown up with a specific conglomerate owning, not just the mall that you go to, but also the apartment building that you live in, and the school that you go to.

因此，我们是在一家特定的企业集团的陪伴下成长起来的，不仅拥有您去的购物中心，还拥有您居住的公寓楼和您就读的学校。

And so they're very well integrated and there's this sense of trust in a conglomerate.

因此，它们整合得非常好，并且对一个企业集团有这种信任感。

Whereas in America we already shy away from, does Google know too much about me?

而在美国我们已经回避了，谷歌是否对我了解太多？

There's also, I think, the second aspect of it, which is that in Asia, we've kind of leapfrogged to the computer era.

我认为还有第二个方面，那就是在亚洲，我们已经跨越到了计算机时代。

So everyone has a phone, but you may not even have a computer in the entire household.

所以每个人都有电话，但你可能整个家庭都没有电脑。

And so when your phone is full, are you going to delete a photo of your kid or are you going to delete this app?

那么当你的手机已满时，你会删除你孩子的照片还是会删除这个应用程序？

You're probably going to delete the app.

您可能会删除该应用程序。

So for anyone to really survive, it has to be part of this super app concept.

因此，对于任何人来说，要想真正生存下来，它就必须成为这个超级应用程序概念的一部分。

Lenny: Oh wow.

莱尼：哦，哇。

I've never thought of it that way.

我从来没有这样想过。

That you don't have a lot of space on your phone and so you want one app to do a lot of things.

您的手机上没有太多空间，因此您希望一个应用程序可以执行很多操作。

Crystal Widjaja: That's right.

Crystal Widjaja：没错。

So there's a decision factor that you don't really have in the US because the cloud storage and device capacity there is a little bit bigger.

因此，有一个在美国实际上没有的决定因素，因为那里的云存储和设备容量要大一些。

Lenny: Interesting.

莱尼：有趣。

So in the US you can have different apps to be...

所以在美国你可以有不同的应用程序......

Basically a super app doesn't have to be the best at everything.

基本上，超级应用程序不必在所有方面都是最好的。

The fact that it does enough and everything good enough.

事实上，它做得足够好，一切都足够好。

Wow, it's fascinating.

哇，真迷人。

Crystal Widjaja: You just need to get the job done.

Crystal Widjaja：你只需要完成工作即可。

Lenny: Amazing.

莱尼：太棒了。

Okay.

好的。

That's super interesting.

这太有趣了。

Okay.

好的。

So transitioning a bit to growth and things you've learned along the way.

因此，稍微过渡一下成长和你一路上学到的东西。

So you talked about how, I think Gojek, you said hired tens of thousands of drivers really quickly.

所以你谈到了，我想 Gojek，你说你如何很快雇佣了数万名司机。

Are there things that startups in Asia do that you think companies in the US should do and can learn from in terms of growth?

您认为亚洲初创企业在增长方面是否应该做并且可以向美国企业学习哪些事情？

Crystal Widjaja: Yeah, so we did crazy things.

Crystal Widjaja：是的，所以我们做了一些疯狂的事情。

If someone told you, in the US, that they were going to rent out a stadium, pre-load a bunch of mobile devices, market that drivers should come here in mass for a job fair.

如果有人告诉你，在美国，他们要租用一个体育场，预装一堆移动设备，然后推销司机应该大批来这里参加招聘会。

They're going to give them a phone and send them on their way, some people would say, "No. That's crazy.

他们会给他们一部电话，然后送他们上路，有些人会说，“不。这太疯狂了。

Won't we get in trouble." And to an extent, maybe that's true.

我们不会遇到麻烦吧。”在某种程度上，也许这是真的。

So maybe there are some limitations there, but this concept of doing things that are somewhat crazy, but validate a point, doing stuff that don't scale, especially I think is really the bread and butter of what we did at Gojek.

所以也许存在一些限制，但是这种做事情的概念有点疯狂，但验证了一个观点，做一些无法扩展的事情，特别是我认为这确实是我们在 Gojek 所做的事情的基础。

We were insanely scrappy.

我们非常斗志旺盛。

Crystal Widjaja: We would do things as simple as wanting to test a subscription feature, which was just released in Singapore a couple weeks ago.

Crystal Widjaja：我们会做一些简单的事情，比如测试订阅功能，该功能几周前刚刚在新加坡发布。

We ended up saying, "We have this voucher system that we can distribute vouchers in the back end.

我们最后说：“我们有这个优惠券系统，我们可以在后端分发优惠券。

We obviously know our driver's phone numbers.

我们显然知道司机的电话号码。

Why don't we just add them to a WhatsApp group?" We'll add a hundred drivers randomly to a WhatsApp group.

我们为什么不将他们添加到 WhatsApp 群组呢？”我们将随机添加一百名司机到 WhatsApp 群组。

We'll tell them, "Every time you are on a ride with a customer, try to sell them this pitch.

我们会告诉他们，“每次你和顾客一起乘车时，试着向他们推销这个产品。

You are the only driver who can sell a subscription package.

您是唯一可以出售订阅套餐的司机。

Have the customer give you $10.

让顾客给你 10 美元。

Text us when they say yes.

当他们答应时给我们发短信。

Someone will be sitting by this phone all day, every day.

每天都会有人整天坐在这部电话旁。

Crystal Widjaja: We'll look up the customer that you were on a ride with in the backend, we'll give them the vouchers in the back end, and then we'll deduct $10 from your balance." It works.

Crystal Widjaja：我们会在后端查找与您一起乘车的客户，我们会在后端给他们优惠券，然后我们会从您的余额中扣除 10 美元。”有用。

It's really this Wizard of Oz experience.

这确实是绿野仙踪的体验。

We don't have to build anything.

我们不必建造任何东西。

I coordinated with a bunch of interns and we were able to validate some of the value prop and conversion rates that we would expect in a subscription service.

我与一群实习生协调，我们能够验证我们在订阅服务中期望的一些价值支柱和转化率。

When we wanted to do a new onboarding screen, but turns out we have lots of engineering work to do, we took a screenshot of the screen as is, and we just had our designer put what the onboarding flow might look like if we had to overlay it on top of the screen.

当我们想做一个新的引导屏幕，但结果发现我们有很多工程工作要做时，我们按原样截取了屏幕截图，然后让我们的设计师将引导流程显示在我们必须将其覆盖在屏幕顶部时可能会是什么样子。

Crystal Widjaja: And we just sent that as an in-app message.

Crystal Widjaja：我们刚刚将其作为应用内消息发送。

And then eventually I think finding stuff that does scale intuitively.

最终我认为找到能够直观扩展的东西。

We knew that we were sending out lots of fake features through things like Typeform surveys.

我们知道我们通过 Typeform 调查等方式发送了很多虚假功能。

Things like a personality quiz can be very easily done through Typeform.

像性格测验这样的事情可以通过 Typeform 轻松完成。

And we realized that if we built in the in-app webpage and we made it easier for us to do a website deployment on our backend side, we wouldn't have to wait for a mobile app release to test some of these new features out that could be done on web.

我们意识到，如果我们内置应用程序内网页，并且可以更轻松地在后端进行网站部署，那么我们就不必等待移动应用程序发布来测试其中一些可以在网络上完成的新功能。

So it's really just like, what is the user experience that we want to create?

所以这实际上就像，我们想要创造的用户体验是什么？

How do we manifest that as quickly as possible?

我们如何尽快体现这一点？

Let's just try that first.

让我们先尝试一下。

Lenny: Going back to the stadium example.

莱尼：回到体育场的例子。

I know you said that you hired a stadium full of people, I didn't realize it was actually a stadium that you-

我知道你说你雇了一个挤满人的体育场，我没有意识到这实际上是一个体育场，你-

Crystal Widjaja: That it was literally a stadium that we rented, like a football field, a couple football fields if I'm not wrong.

Crystal Widjaja：这实际上是我们租用的一个体育场，就像一个足球场，如果我没记错的话，有几个足球场。

It was long lines, boxes of phones and SIM cards.

队伍排得很长，里面装满了手机和 SIM 卡。

So it was a lot of just doing really hard work to get to that scale.

因此，要达到这样的规模需要付出很多努力。

Lenny: Wow.

莱尼：哇。

I know you do a lot of advising too.

我知道你也提供了很多建议。

Do you advise startups to be more scrappy and do things that don't scale?

您是否建议初创公司更加斗志旺盛，做一些无法规模化的事情？

I imagine because in the US the culture is a little different.

我想是因为美国的文化有点不同。

Crystal Widjaja: The only thing better than knowing...

Crystal Widjaja：唯一比知道更好的事情......

If you have data of what your customers are doing, that is the best data you could ever get.

如果您有客户行为的数据，那就是您可以获得的最佳数据。

And so if you don't have a tested hypothesis, if you can't think of a way to run an experiment, then honestly that idea is pretty useless.

因此，如果你没有经过检验的假设，如果你想不出进行实验的方法，那么老实说，这个想法毫无用处。

Maybe it makes sense to the market, to the model, but you could have weird consumer sentiments.

也许这对市场、对模型来说是有意义的，但你可能会有奇怪的消费者情绪。

Not everyone is a rational actor.

并不是每个人都是理性的演员。

So testing the actual experience and seeing how people respond to it, that's the best possible data.

因此，测试实际体验并看看人们对此有何反应，这就是最好的数据。

Lenny: Pulling that thread a little bit, for startups, experiments are often hard because there's just not enough data and enough users.

Lenny：稍微拉扯一下，对于初创公司来说，实验通常很困难，因为没有足够的数据和足够的用户。

How do you think startups should approach that?

您认为初创公司应该如何解决这个问题？

Can you run experiments when you're really, really early?

你能在非常非常早的时候进行实验吗？

Crystal Widjaja: You should.

Crystal Widjaja：你应该这样做。

Even if you have a sample size of 30, the data you get back, generally, does not change but its precision will.

即使您的样本量为 30，您返回的数据通常不会改变，但其精度会改变。

So mathematically speaking, you're going to get the same level of trends, but the precision at which you understand those trends will become more deep if you have more data.

因此，从数学上讲，您将获得相同水平的趋势，但如果您拥有更多数据，您对这些趋势的理解精度将变得更加深入。

But the underlying information that you're getting out of that won't be very different at larger scales.

但从更大的范围来看，你从中获得的基本信息不会有太大不同。

So what's better than having 30 data points?

那么还有什么比拥有 30 个数据点更好呢？

Certainly having 100.

当然有100个。

But what's better than having zero is definitely 30.

但比零更好的绝对是 30。

Lenny: Fascinating.

莱尼：令人着迷。

So contrarian.

如此逆反。

Running experiments at 30 people.

对 30 人进行实验。

I love that.

我喜欢那个。

Crystal Widjaja: You have to.

Crystal Widjaja：你必须这样做。

Every idea is so cheap at that scale.

在这种规模下，每个想法都很便宜。

You could do things that don't scale dramatically better with 30 people than at 100 if you're testing.

如果您正在进行测试，那么 30 人的规模并不比 100 人的规模大得多。

Lenny: Just to pull on that a little bit, when you're running an experiment with 30 people, what do you look for?

Lenny：简单来说，当你对 30 个人进行实验时，你会寻找什么？

You're looking for 20 of them to do something, a large percentage of that group does something?

您正在寻找 20 个人来做某事，该组中的大部分人都做了某事？

Crystal Widjaja: So everyone wants to go on retention.

Crystal Widjaja：所以每个人都想继续保留。

They want to see that users are doing this thing, and they want to get from step zero to 100 really quickly, but they don't realize that users make decisions based on succeeding events.

他们希望看到用户正在做这件事，并且希望快速从第 0 步到第 100 步，但他们没有意识到用户是根据后续事件做出决策的。

So what's one step before the user makes that decision?

那么用户做出决定之前需要做什么？

What are the things that they have to do, the things that have to be done?

有哪些事情是他们必须做的、必须做的？

So we're always looking for what is a specific reason that this user might have converted?

因此，我们一直在寻找该用户可能发生转化的具体原因是什么？

For things like GoFood it would be things like when does a user try a new merchant if what people are ordering right now or just food that they already trust and know.

对于像 GoFood 这样的东西，它会是这样的事情：用户什么时候尝试新的商家，人们现在订购什么，或者只是他们已经信任和知道的食物。

If you need to have trust in order to purchase food from a merchant, how do we generate that trust?

如果您需要信任才能从商家购买食品，我们如何产生这种信任？

So we actually hacked it by connecting people's Facebook connect login.

所以我们实际上是通过连接人们的 Facebook 连接登录来破解它的。

Crystal Widjaja: So we had already had permission to look at who they had connected with on Facebook.

Crystal Widjaja：所以我们已经获得许可查看他们在 Facebook 上与谁联系过。

We actually looked at the food that their friends had purchased and used that as a data set of, "Hey, here's food that Lenny purchased and liked.

我们实际上查看了他们的朋友购买的食物，并将其用作数据集，“嘿，这是莱尼购买并喜欢的食物。

Maybe you would like it too." And so that was one way to hack the trust factor.

或许你也会喜欢。”所以这是破解信任因素的一种方法。

And we did find that when we told people, "This friend purchased from this merchant," you would be twice as likely to purchase from a brand new restaurant then users who did not have this feature.

我们确实发现，当我们告诉人们“这位朋友从这个商家购买商品”时，您从一家全新餐厅购买商品的可能性是没有此功能的用户的两倍。

And that increases GMV, that eventually gets you to the conversion rate that you wanted, but it solved a different problem.

这会增加 GMV，最终让你达到你想要的转化率，但它解决了另一个问题。

Before how do I convert, it was how do I solve for trust?

在如何转化之前，是如何解决信任？

How do I break the barrier of facilitating that decision making process, that aha moment, by fixing the setup moment, which was trust?

我如何通过修复建立时刻（即信任）来打破促进决策过程（即顿悟时刻）的障碍？

Lenny: And that's just a general rule of thumb you have.

莱尼：这只是你的一般经验法则。

Don't use retention as a goal.

不要将保留率作为目标。

I know you wrote about this somewhere.

我知道你在某处写过这个。

Is that a rough rule of thumb you use?

这是您使用的粗略经验法则吗？

Crystal Widjaja: I think a lot of people thought that I had meant retention sucks, don't care about it at all.

Crystal Widjaja：我认为很多人认为我的意思是保留率很糟糕，根本不关心它。

But in reality it was really when you think about retention, that's just not specific enough.

但实际上，当你考虑保留时，这确实不够具体。

So there is this mental model that I use from made to stick where they'll tell you like, "Lenny, think of everything in the world that is orange." And you're like, "An orange.

所以我用了一种心理模型来坚持他们会告诉你的地方，“莱尼，想想世界上所有的东西都是橙色的。”你就像，“一个橙子。

What else?" And then if you change that structure with sandbox to think of everything orange that's in a construction site, then you really start to realize and grasp at concrete concepts, and can actually action on them in real life.

还有什么？”然后，如果你用沙箱改变这种结构，将建筑工地中的所有橙色事物都考虑在内，那么你就真正开始意识到并掌握具体的概念，并可以在现实生活中对它们采取实际行动。

Lenny: Got it.

莱尼：明白了。

Speaking of retention, where have you found products and companies have the most success increasing retention?

说到保留率，您发现哪些产品和公司在提高保留率方面最成功？

Crystal Widjaja: It's usually the step right before conversion.

Crystal Widjaja：这通常是转换之前的步骤。

So if they aren't sure why the user opens the app or they aren't sure why the user got to this checkout page, it's often some copy or the path has been ineffective in some way.

因此，如果他们不确定用户为什么打开应用程序，或者不确定用户为什么进入此结帐页面，则通常是某些副本或路径在某种程度上无效。

I'd like to see founders think about the user psych model that Darius Contractor often talks about.

我希望看到创始人思考 Darius Contractor 经常谈论的用户心理模型。

So you need some momentum in that user journey to get them over the hump of some of these very painful user processes like typing in a credit card.

因此，您需要在用户旅程中提供一些动力，帮助他们克服一些非常痛苦的用户流程（例如输入信用卡信息）。

That's a lot of work.

这是很多工作。

How do you lower that friction?

如何降低摩擦力？

And being able to sequence the right steps effectively and just moving around screens actually can do a lot.

能够有效地排序正确的步骤并在屏幕上移动实际上可以做很多事情。

Lenny: Going even deeper there.

莱尼：再深入一点。

So the companies you've worked at, the companies you've advised, you're on the boards of a couple companies I noticed, what have you found to be really good uses of time in terms of growth investments, things that often work?

那么，对于你工作过的公司、你提供过建议的公司、你在我注意到的几家公司的董事会中任职的公司，你发现在增长投资方面真正有效利用时间的是什么，通常是有效的？

And then a second question, what do you find is rarely successful or people invest a lot of time and ends up not being really useful for growth?

然后是第二个问题，你发现什么很少成功，或者人们投入了大量时间，但最终对增长并没有真正有用？

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

I think I see a lot of founders grasping at straws.

我想我看到很多创始人都在抓住救命稻草。

So there'll be this brand new feature that does something different from what people are already doing on our app, like this will make things work.

因此，将会有一个全新的功能，它会做一些与人们已经在我们的应用程序上做的事情不同的事情，就像这样会让事情顺利进行。

But they don't have any Wizard of Oz test, they haven't proven that people want to do that, they don't have any data of users currently trying to do that.

但他们没有任何绿野仙踪测试，他们没有证明人们想要这样做，他们没有当前尝试这样做的用户的任何数据。

And that's a sign of why this, instead of literally anything else that you could be doing.

这就是为什么要这样做，而不是你可以做的任何其他事情的标志。

I do find if you have a lot of people landing on a webpage or an app and then not doing anything, then it's probably copy.

我确实发现，如果有很多人登陆网页或应用程序，然后没有执行任何操作，那么它可能是复制的。

Crystal Widjaja: They haven't even experienced the product, it's clearly not the product that's wrong.

Crystal Widjaja：他们甚至没有体验过该产品，显然不是产品出了问题。

So how can you change the copy and resonate with the pain point rather than the solution you are offering so that users understand how to fit themselves into the use case?

那么，您如何更改文案并与痛点而不是您提供的解决方案产生共鸣，以便用户了解如何使自己适合用例？

So copy is a big one if I see conversion rates aren't landing between app launch to some first action.

因此，如果我发现应用程序启动到第一次操作之间转化率没有下降，那么文案就是一个大问题。

But if there is conversion and they're just not as frequent, I try to look at what the most painfully long conversion events are.

但如果有转化，而且转化频率不那么频繁，我会尝试看看最痛苦的长转化事件是什么。

So users who eventually check out or eventually completed the aha moment, what are the user paths, and what is the longest one that seems like it's the most painful?

那么最终签出或最终完成顿悟时刻的用户，用户路径是什么，最长且看起来最痛苦的路径是什么？

Are there enough people trying to do that?

有足够多的人尝试这样做吗？

Crystal Widjaja: And how do we shorten that cycle?

Crystal Widjaja：我们如何缩短这个周期？

So for Kumu things like users wanted to sign up and find their friends on Kumu.

因此，对于 Kumu 来说，用户想要在 Kumu 上注册并找到他们的朋友。

And so they were using search frequently, search was underutilized API, it was slow.

因此，他们经常使用搜索，但搜索 API 未得到充分利用，速度很慢。

We sped that up.

我们加快了速度。

Conversion rates go from 60% to 90% over the course of a few weeks of just optimizing that and putting more content there.

经过几周的优化和添加更多内容，转化率从 60% 上升到 90%。

So looking at where are people doing things and then failing, you already know this percent of people would convert if you fixed this, that's a definite potential win.

因此，看看人们在哪里做事然后失败，你已经知道如果你解决这个问题，这个百分比的人会转变，这绝对是潜在的胜利。

So we try to layer these definite wins with crazy bets of brand new feature with no data.

因此，我们尝试在没有数据的情况下，通过对全新功能的疯狂押注来获得这些明确的胜利。

At least run an experiment if you can.

如果可以的话，至少进行一次实验。

But I always try to layer in these sure wins.

但我总是尝试将这些确定的胜利分层。

Lenny: When you talk about conversion being good and bad, do you have a rule of thumb or of a mental model of here's a rough range of this is good and we should not really spend a lot of time on this and this is bad and we should optimize?

Lenny：当您谈论转化的好与坏时，您是否有一个经验法则或心理模型，其中有一个粗略的范围，这是好的，我们不应该在这上面花很多时间，这是坏的，我们应该优化？

Crystal Widjaja: So assuming that the frequency is correct, so you have a weekly frequency, if users are coming back, if it's a free product, 60%.

Crystal Widjaja：假设频率是正确的，那么如果用户回来，如果它是免费产品，那么你每周的频率是 60%。

It has to be at least 60%.

它必须至少为 60%。

If it's a free product, we go over a week.

如果它是免费产品，我们会花一周多的时间。

If it's a paid product, I usually look at that more as maybe 20 to 30%.

如果是付费产品，我通常会考虑 20% 到 30%。

Lenny: And this is retention, people coming back the next week?

莱尼：这就是保留，人们下周会回来吗？

Crystal Widjaja: Exactly.

克里斯特尔·维贾贾：确实如此。

Coming back in the second week or month or whenever your frequency ratio is.

在第二周或第二个月或您的频率比率时回来。

And this is at scale.

这是大规模的。

So if you are much smaller, your friends and family that better be near close to 80% no matter what, because if you can't even convince the people who care about you to use the product, it probably isn't going to solve the job for anyone else.

因此，如果您的规模要小得多，那么无论如何，您的朋友和家人最好接近 80%，因为如果您甚至无法说服关心您的人使用该产品，那么它可能无法解决其他任何人的工作。

Lenny: Very handy.

莱尼：非常方便。

Very concrete numbers.

非常具体的数字。

And then your point is that when you're a startup, it's only going to go down because your early adopters that are more excited and they'll be more excited by coming back.

然后你的观点是，当你是一家初创公司时，它只会下降，因为你的早期采用者更加兴奋，并且他们回来后会更加兴奋。

So you want to start really high.

所以你想从很高的起点开始。

Crystal Widjaja: Don't make the same mistake that Netflix and Spotify have made, which I guess is when they've launched, they've started international expansion and they see this very small percentage of users start to sign up for Spotify or Netflix.

Crystal Widjaja：不要犯 Netflix 和 Spotify 所犯的同样错误，我猜当他们推出时，他们已经开始国际扩张，他们看到只有极少数的用户开始注册 Spotify 或 Netflix。

There are very few people though in Southeast Asia or internationally that have the types of credit cards that Spotify or Netflix would accept.

尽管在东南亚或国际上很少有人拥有 Spotify 或 Netflix 接受的信用卡类型。

And so when they launch in these markets and they see a ton of uptick in the first week, they're like, "This is only going to get better." When in reality it's like you just pulled forward everyone who could have possibly subscribed to you, now you're going to have to work a lot harder to get everyone else.

因此，当他们在这些市场推出产品并在第一周看到大量增长时，他们会想，“这只会变得更好。”实际上，就像您刚刚吸引了所有可能订阅您的人一样，现在您将不得不更加努力地工作才能吸引其他人。

Lenny: The 60% number.

莱尼：60%的数字。

So you're saying it's then every week, 60% of the previous week come back, roughly.

所以你是说，然后每周，大约 60% 的前一周都会回来。

Is it just a rule of thumb?

这只是经验法则吗？

Lenny: Is that how you think about it versus say cohort retention?

莱尼：与群体保留相比，你是这么看待这个问题的吗？

Is that just because it's easier is just a simple rule of thumb?

这仅仅是因为它更容易只是一个简单的经验法则吗？

Crystal Widjaja: Am actually thinking of it as cohorts.

Crystal Widjaja：我实际上将其视为群组。

So 60% should be your week one, and then it should flatten.

所以 60% 应该是你的第一周，然后它应该趋于平缓。

I think I usually give teams two to three weeks or frequency periods to see things flatten, but it better flatten around 60% for a free product.

我想我通常会给团队两到三周或频率周期来让事情变得平坦，但对于免费产品来说，最好平坦化 60% 左右。

That's actually what we saw at Gojek.

这实际上就是我们在 Gojek 看到的情况。

Early days it was like 60, 70% retention rates because people were using this product that really solved a huge problem for them.

早期的保留率约为 60%、70%，因为人们使用的产品确实为他们解决了一个巨大的问题。

And I think that's when I knew we were going to be fine.

我想那时我就知道我们会没事的。

If people keep coming back, the product just needs to work.

如果人们不断回来，那么产品就需要发挥作用。

Lenny: Wow.

莱尼：哇。

So week one, 40% of people drop off week two and beyond basically nobody drops off is what you look for.

因此，第一周，40% 的人离开，第二周及之后基本上没有人离开，这就是你所寻找的。

Lenny: Wow.

莱尼：哇。

What a high bar.

多么高的门槛啊。

But I like that, because-

但我喜欢这样，因为——

Crystal Widjaja: Yeah, well Gojek is a decacorn.

Crystal Widjaja：是的，Gojek 是一个十角兽。

Lenny: Okay.

莱尼：好的。

There we go.

我们开始吧。

If you want to be a decacorn, there's your new benchmark.

如果你想成为十角兽，这就是你的新基准。

Amazing.

惊人的。

Okay.

好的。

There's a bunch of other stuff I want to dig into.

我还有很多其他东西想深入研究。

One is just data modeling and thinking about growth strategy as a founder.

一是作为创始人进行数据建模和思考增长策略。

So say a startup is just trying to think about, how do we drive growth, where do we invest, do you have a framework or a process?

假设一家初创公司只是想思考，我们如何推动增长，我们在哪里投资，你们有框架或流程吗？

I know this might be a really big question, but just for founders to think about how their growth works, what their drivers might be, how would a founder approach that problem?

我知道这可能是一个非常大的问题，但只要创始人思考一下他们​​的增长是如何运作的，他们的驱动因素可能是什么，创始人将如何解决这个问题？

Crystal Widjaja: For sure.

Crystal Widjaja：当然。

So I thought that this was not an obvious process.

所以我认为这不是一个显而易见的过程。

It wasn't an explicit process until I worked with Reforge to build my data four PMs program.

直到我与 Reforge 合作构建我的数据四 PM 程序之前，这并不是一个明确的过程。

Got to get that plug there.

必须把那个插头拿到那里。

Crystal Widjaja: I basically talked with the Reforge folks about here's what I would do in all of these scenarios.

Crystal Widjaja：我基本上与 Reforge 人员讨论了在所有这些情况下我会做什么。

And they're like, "Oh, so you mean you're doing this step one, step two?" And I was like, "Yes, actually.

他们就像，“哦，那么你的意思是你正在做第一步，第二步？”我当时想，“是的，实际上。

How did you figure that out?" So I don't really think in-frame works, this is just a logical process to me.

你是怎么想出来的？”所以我真的不认为框架内有效，这对我来说只是一个逻辑过程。

But I think what I've figured out is, it's step one, you have constraints.

但我认为我发现的是，这是第一步，你有限制。

Similar to our sandbox example of everything in the world that's orange versus everything in a construction site, you have to think about the physics of the current market, the product, the model and the channels that you're using.

类似于我们的沙盒示例，世界上所有橙色的东西与建筑工地中所有东西的对比，你必须考虑当前市场的物理情况、产品、模型和你正在使用的渠道。

So to use Gojek as an example, it would be market of Indonesia.

以Gojek为例，那就是印度尼西亚市场。

Crystal Widjaja: Here are the consumers in this market, the driver's side supply side in this market.

Crystal Widjaja：这是这个市场的消费者，这个市场的司机方、供应方。

Here is the product, mobile app.

这是产品，移动应用程序。

We're able to connect drivers and consumers.

我们能够连接司机和消费者。

There is a allocation that we create model.

我们创建模型有一个配置。

We charge per order, channel.

我们按订单、渠道收费。

We are able to do this through push notifications or in acquiring new users.

我们可以通过推送通知或获取新用户来做到这一点。

It might be through Facebook ads, or, and this was a really big insight for us, it's the real world.

可能是通过 Facebook 广告，或者，这对我们来说是一个非常重要的洞察，这就是现实世界。

There was a physical conception of a driver in a jacket driving around the city who was marketing Gojek for us.

有一个穿着夹克的司机在城市里开车为我们推销 Gojek 的物理概念。

And word of mouth actually was primarily driven by, "I saw a driver on the street, so I knew Gojek was here."

口碑实际上主要是由“我在街上看到一个司机，所以我知道 Gojek 在这里”驱动的。

Crystal Widjaja: And that actually was a huge driver of all of Gojek's growth as it expanded to new cities.

Crystal Widjaja：这实际上是 Gojek 向新城市扩张时所有增长的巨大推动力。

So step one is, what are the physics?

所以第一步是，什么是物理原理？

Step two is when you think about loops and growth funnels and the quantitative inputs to each loop, does that fit into these physics or do you have to change four or five different things?

第二步是当你考虑循环和增长漏斗以及每个循环的定量输入时，这是否符合这些物理原理，或者你是否必须改变四到五个不同的东西？

So we were very careful about changing too many parameters and making too many bets on too many variables going our way.

因此，我们非常小心地避免改变太多的参数，并对太多的变量进行太多的押注。

So we would always change one small thing at a time and make sure that it fit into the model.

所以我们总是一次改变一件小事，并确保它适合模型。

Lenny: This episode is brought to you by Eppo.

Lenny：这一集是 Eppo 为您带来的。

Eppo is a next generation AB testing platform built by Airbnb alums for modern growth teams.

Eppo 是 Airbnb 校友为现代增长团队打造的下一代 AB 测试平台。

Companies like Netlify, Contentful and Cameo rely on Eppo to power their experiments.

Netlify、Contentful 和 Cameo 等公司依靠 Eppo 来支持他们的实验。

Wherever you work, running experiments is increasingly essential, but there are no commercial tools that integrate with a modern growth team stack.

无论您在哪里工作，进行实验都变得越来越重要，但是没有可以与现代增长团队堆栈集成的商业工具。

This leads to waste of time building internal tools or trying to run your experiments through a clunky marketing tool.

这会导致浪费时间构建内部工具或尝试通过笨重的营销工具进行实验。

When I was at Airbnb, one of the things that I loved about our experimentation platform was being able to easily slice results by device, by country and by user stage.

当我在 Airbnb 工作时，我喜欢我们的实验平台的一件事是能够轻松地按设备、国家和用户阶段划分结果。

Lenny: Eppo does all that and more.

Lenny：Eppo 做到了这一切，甚至更多。

Delivering results quickly, avoiding annoying prolonged analytics cycles and helping you easily get to the root cause of any issue you discover.

快速提供结果，避免恼人的长时间分析周期，并帮助您轻松找到所发现问题的根本原因。

Eppo lets you go beyond basic clickthrough metrics and instead uses our north star metrics like activation, retention, subscriptions and payments.

Eppo 让您超越基本的点击率指标，而是使用我们的北极星指标，例如激活、保留、订阅和付款。

Eppo supports test on the front end, the back end, email marketing and even machine learning clients.

Eppo支持前端、后端、电子邮件营销甚至机器学习客户端的测试。

Check out Eppo at geteppo.com.

请访问 geteppo.com 查看 Eppo。

Geteppo.com and 10 X your experiment velocity.

Geteppo.com 和 10 倍的实验速度。

So step one, just to cover this, is figure out how you're growing.

因此，第一步，就是要弄清楚你是如何成长的。

In Gojek's case, it was partly real world, people just seeing Gojek riding around.

就 Gojek 而言，这部分是现实世界，人们只是看到 Gojek 骑着马到处走。

Crystal Widjaja: I think it's both figure out how you're growing and also the elements that you have at your disposal.

Crystal Widjaja：我认为这既要弄清楚你如何成长，也要弄清楚你可以使用的元素。

What are the levers that you have that maybe you've never tried using?

您拥有哪些可能从未尝试过使用的杠杆？

When we looked at our model this way, we actually realized we had underutilized the driver's capacity to drive our growth.

当我们以这种方式审视我们的模型时，我们实际上意识到我们没有充分利用驾驶员的能力来推动我们的增长。

Pun definitely intended.

双关语绝对是故意的。

So in looking at the model this way, we had thought through what is our goal.

因此，在以这种方式看待模型时，我们思考了我们的目标是什么。

We want GoPay to be much bigger than it really is.

我们希望 GoPay 的规模比实际规模大得多。

It's a E-wallet service, users are able to get access to this digital balance.

这是一项电子钱包服务，用户可以访问该数字余额。

How do we drive adoption?

我们如何推动采用？

And so when we looked at the lever of we have a driver, we actually created an incentive model.

因此，当我们考虑拥有司机的杠杆时，我们实际上创建了一个激励模型。

So we built a very small service that would check when a driver got allocated to a customer, again, the product and the model, we would then check in the database, has this customer ever used our GoPay product before?

因此，我们建立了一个非常小的服务，可以检查司机何时分配给客户，以及产品和型号，然后我们会检查数据库，该客户以前是否使用过我们的 GoPay 产品？

Crystal Widjaja: Did they have a digital balance?

Crystal Widjaja：他们有数字天平吗？

And if the answer was no, we would message the driver immediately, "Hey, this customer hasn't done a GoPay top up before.

如果答案是否定的，我们会立即向司机发送消息，“嘿，这位顾客以前没有进行过 GoPay 充值。

If you get them to give you cash and we deposit it into their virtual wallet, we'll give you extra money." So using them as the salesperson.

如果你让他们给你现金，然后我们将其存入他们的虚拟钱包，我们就会给你额外的钱。”所以用他们作为销售人员。

You wouldn't believe how great of a salesperson someone can be when you were literally trapped in a car with them going somewhere.

当你真的被困在车里和他们一起去某个地方时，你不会相信某人能成为多么出色的销售人员。

And so you have this captive audience, captive attention, you have someone who has the incentive to cross pay or cross sell someone into GoPay.

因此，你拥有了这些被俘获的受众、被俘获的注意力，你有一个有动力交叉支付或交叉销售某人到 GoPay 的人。

And customers were able to feel the benefit because the driver was explaining it to them directly.

客户能够感受到好处，因为司机直接向他们解释。

There was no change to the physics, it was a lever usage.

物理原理没有改变，只是杠杆的使用。

Crystal Widjaja: It was huge.

Crystal Widjaja：它很大。

It was 60% of acquisition once we released that.

当我们发布该信息时，该数字已占收购量的 60%。

Lenny: Oh my God.

莱尼：天啊。

So for thinking through your potential levers and physics of your growth, do you think about it bottoms up, here's all the things that are going on and here's areas we can invest?

因此，在思考你的潜在杠杆和增长物理过程时，你是否会自下而上地思考，这是正在发生的所有事情以及我们可以投资的领域？

Or do you have a menu of options top down of here's the 10 things it could be, it's looking like for Gojek it's these four and let's focus on that?

或者你有一个从上到下的选项菜单，其中可能有 10 件事，对于 Gojek 来说就是这四件事，让我们重点关注这一点？

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

I think you always have to start from the fact that we are not wizards.

我认为你总是必须从我们不是巫师这一事实开始。

It's very hard to move the physics of a universe when you are trying these new things.

当你尝试这些新事物时，很难改变宇宙的物理原理。

So start with what currently works and currently exists and where you think the biggest constraint is or the best lever is, and then fix that one piece because the entire universe isn't exploding.

因此，从当前有效和当前存在的内容以及您认为最大的限制或最佳杠杆所在开始，然后修复该部分，因为整个宇宙不会爆炸。

The world isn't changing so dramatically that your physics change.

世界并没有发生如此巨大的变化，以至于你的物理学发生了变化。

So I think rooted in reality is very important.

所以我觉得植根于现实非常重要。

Lenny: Got it.

莱尼：明白了。

Okay.

好的。

So it's see what's working, find the constraints.

所以要看看什么在起作用，找到限制。

And then step two is basically what can you do to the product to optimize the funnel/loop to make it go even faster?

第二步基本上是你可以对产品做些什么来优化漏斗/循环，使其运行得更快？

Lenny: Love that.

莱尼：喜欢这个。

Maybe as another example, if something comes to mind, with Kumu, how do you think of Kumu through this lens?

也许再举个例子，如果你想到 Kumu，你通过这个镜头如何看待 Kumu？

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

I'm always very hesitant to talk about Kumu, because there's so much competition right now and we are on the cusp of some very interesting things.

我总是很犹豫是否要谈论 Kumu，因为现在竞争非常激烈，而且我们正处于一些非常有趣的事情的风口浪尖。

But I think for Kumu, it's actually very complex because there's a lot of human emotion that is involved.

但我认为对于 Kumu 来说，它实际上非常复杂，因为涉及到很多人类情感。

With Gojek you knew if you got the job done.

有了 Gojek，您就知道自己是否完成了工作。

You made a transaction.

你做了一笔交易。

With Kumu, how do you know if a consumer made a friend, felt like they had a genuine friendship?

有了 Kumu，你怎么知道消费者是否交了朋友，是否感觉他们有真正的友谊？

So you almost have to create more friction to identify users who really got past that barrier and aren't explicit with the activity that they did.

因此，您几乎必须制造更多摩擦来识别真正克服了这一障碍并且对他们所做的活动不明确的用户。

So we have features that tell us if a user is really searching for this job to be done, if they really want to be part of a community, how do they fill out this?

因此，我们有一些功能可以告诉我们用户是否真的在寻找要完成的工作，如果他们真的想成为社区的一部分，他们如何填写这个？

Crystal Widjaja: Do they fill out the form?

Crystal Widjaja：他们填写表格吗？

Do they fill out a questionnaire of many questions?

他们是否填写了包含许多问题的调查问卷？

Do they go through this friction just to get access to a community?

他们经历这种摩擦只是为了进入社区吗？

So we almost create this artificial friction to help differentiate how deeply a user wants something or needs something.

因此，我们几乎创造了这种人为的摩擦，以帮助区分用户想要某物或需要某物的深度。

And if the user doesn't fill out that questionnaire, maybe they're actually looking for something else.

如果用户没有填写该调查问卷，也许他们实际上是在寻找其他东西。

They were looking for entertainment.

他们正在寻找娱乐。

They were looking for content or short form content.

他们正在寻找内容或简短的内容。

And so creating almost like hand razor approaches for a user to say, "I wanted this thing." We leave a lot of breadcrumbs in the app to be able to identify those paths.

因此，创建几乎类似于手动剃须刀的方法，让用户说：“我想要这个东西。”我们在应用程序中留下了很多面包屑，以便能够识别这些路径。

Lenny: Awesome.

莱尼：太棒了。

While we're on the topic of these two companies, just maybe for inspiration to founders who are thinking of ways to drive growth, what were a couple of the bigger unlocks growth wise for these two companies or even any other company that you've worked with that's interesting?

当我们讨论这两家公司的话题时，也许只是为了给那些正在思考如何推动增长的创始人带来启发，那么对于这两家公司甚至您合作过的任何其他公司来说，有哪些更大的增长智慧？

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

Definitely, in the early days it was copy.

当然，在早期它是复制的。

So I think if your product does something that's not super familiar, you have to tie it to something that is.

所以我认为，如果你的产品做了一些不太熟悉的事情，你就必须将它与熟悉的东西联系起来。

So I talked about using drivers to sell GoPay.

所以我就谈到了利用司机来销售GoPay。

Before that, one thing that we did was to actually take someone's virtual account number and put it onto a picture of a credit card.

在此之前，我们所做的一件事就是实际获取某人的虚拟帐号并将其放在信用卡的图片上。

You know what a credit card is, that's familiar to you.

您知道什么是信用卡，这对您来说很熟悉。

A lot of people didn't know what a digital wallet was.

很多人不知道什么是数字钱包。

And so when they looked at this like, "Oh, okay.

所以当他们看到这个的时候，“哦，好吧。

I have this virtual thing that acts like a credit card.

我有一个像信用卡一样的虚拟东西。

It works like my debit account." Then they understood the concept a lot better.

它的运作方式就像我的借记账户一样。”然后他们对这个概念有了更好的理解。

Crystal Widjaja: And we actually saw top ups increase based on us, literally just sending that picture with someone's virtual account number there.

Crystal Widjaja：实际上，我们看到充值量随着我们的增加而增加，实际上只是将带有某人虚拟帐号的图片发送到那里。

So they could go to an ATM and they would just type in the card number as they would a regular debit account.

因此，他们可以去 ATM 机，只需输入卡号，就像输入常规借记账户一样。

And they realized that they could top up through that channel.

他们意识到他们可以通过该渠道充值。

Because that was something that was pretty interesting to us, with just how do we tie the familiarity loop back into the consumer mental model of the product and drive acquisition that way?

因为这对我们来说非常有趣，我们如何将熟悉度循环与产品的消费者心智模型联系起来并以这种方式推动购买？

Lenny: Is there anything else maybe, since you don't want to talk too much about Kumu, any other advisorships or companies, examples of something that ended up working really well to help them accelerate growth?

Lenny：既然您不想过多谈论 Kumu、任何其他咨询机构或公司，还有其他什么事情最终能很好地帮助他们加速增长的例子吗？

Crystal Widjaja: Things that have worked really well.

Crystal Widjaja：效果非常好。

So for one of the companies I work with, AB&B, they run a lot of their D2C brands in South America and globally.

因此，对于我合作的公司之一 AB&B，他们在南美和全球运营着许多 D2C 品牌。

So one of the features that we were looking at was how do we ensure that subscriptions don't actually become a canceling point for a user.

因此，我们正在研究的功能之一是如何确保订阅实际上不会成为用户的取消点。

So in the app you could cancel or you could resume your subscription, but you couldn't pause it.

因此，在应用程序中，您可以取消或恢复订阅，但无法暂停。

So when we looked at the cancellation reasons and we saw that their number one reason was, I still have too much fear, we actually decided, well, let's just add a pause button then.

因此，当我们查看取消原因时，我们发现他们的首要原因是，我仍然有太多的恐惧，我们实际上决定，好吧，让我们添加一个暂停按钮。

Crystal Widjaja: Because canceling the subscription is a permanent solution to having too much fear.

Crystal Widjaja：因为取消订阅是解决太多恐惧的永久解决方案。

How do you make a temporary solution that solves the actual problem?

如何制定临时解决方案来解决实际问题？

Adding in a pause button actually helped alleviate a lot of the churn that was becoming very hard to reacquire back.

添加暂停按钮实际上有助于减轻很多很难重新恢复的流失。

So that was one fix where we looked at the, again, physics of the model.

因此，这是我们再次研究模型物理原理的一个解决方案。

We're not going to create new changes to the product or create one time buys or reactivation emails.

我们不会对产品进行新的更改，也不会创建一次性购买或重新激活电子邮件。

We'll just solve the problem at that small constraint where everyone drops off.

我们将在每个人都退出的小约束下解决问题。

Lenny: Wait, so can you order beer as subscription?

莱尼：等等，你可以订购啤酒作为订阅吗？

Is that a thing?

这是一件事吗？

Is this a consumer product or is this-

这是消费品还是——

Lenny: Cool.

莱尼：酷。

Okay.

好的。

This also reminds me, at Airbnb, this was actually one of the biggest wins, is adding a snooze feature to your listing.

这也提醒我，在 Airbnb，这实际上是最大的胜利之一，就是在您的房源中添加小睡功能。

Exactly the same thing.

完全一样的事情。

Yeah.

是的。

All right there we go.

好吧，我们走吧。

Awesome.

惊人的。

Tip for folks that have churn problems, snooze/pause.

对于有流失问题的人来说，建议暂停/暂停。

I want to shift a little bit to a post that you wrote that maybe is one of your more popular posts you wrote on the Reforge blog called Why Most Analytics Efforts Fail.

我想稍微转移一下您写的一篇文章，这可能是您在 Reforge 博客上写的最受欢迎的文章之一，名为“为什么大多数分析工作都会失败”。

And I'd love to hear your broad overview of why do most analytics efforts fail and then how do teams avoid this?

我很想听听您对为什么大多数分析工作失败以及团队如何避免这种情况的广泛概述？

Maybe what are two to three things they can do?

也许他们能做的两到三件事是什么？

Crystal Widjaja: Yeah, I'm actually pretty surprised at how much noise that has generated because I guess it came from a place of frustration where I kept telling people like, "You are doing this wrong.

Crystal Widjaja：是的，我实际上对产生的噪音感到非常惊讶，因为我猜它来自于一个沮丧的地方，我一直告诉人们，“你做错了。

Here's how you should probably be doing it." But I think it resonated a lot with folks because they recognize all of those symptoms, but they weren't sure why it was happening.

这就是你应该如何做的。”但我认为它引起了人们的强烈共鸣，因为他们认识到所有这些症状，但他们不确定为什么会发生。

So to say, oh, this is the thing, instrumentation is what's wrong, I think it's a very actionable thing.

所以说，哦，这就是问题，仪器仪表就是出了问题，我认为这是一个非常可行的事情。

It's probably one of the most solvable problems out there.

这可能是最容易解决的问题之一。

It just takes some time and mental model shifts to do it well.

只需要一些时间和思维模式的转变就能做好。

So a lot of people look at tracking data as how do I track my OKR?

因此，很多人将跟踪数据视为如何跟踪我的 OKR？

How do I know if I'm going up or down?

我怎么知道我是上升还是下降？

But they don't use it to track or identify insights.

但他们不会用它来跟踪或识别见解。

So I will use the example of using Twitter for "news" when in reality they're actually using Twitter for entertainment.

因此，我将使用使用 Twitter 来获取“新闻”的示例，而实际上他们使用 Twitter 是为了娱乐。

Crystal Widjaja: Do not treat metric gathering as entertainment.

Crystal Widjaja：不要将度量收集视为娱乐。

It's not there for you to be like, "Oh, that's interesting, how novel," and then not act on it.

你不会说，“哦，这很有趣，多么新颖”，然后就不采取行动。

So real news is information that changes what you do in the real world.

所以真正的新闻是改变你在现实世界中所做的事情的信息。

And if you don't change what you're doing, what you are doing is just getting entertainment.

如果你不改变你正在做的事情，你所做的只是为了娱乐。

So let's use that as the premise.

那么我们以此为前提。

The next step in instrumentation is to look at the fact that measurements do not equate to insights.

仪器仪表化的下一步是审视测量并不等于洞察这一事实。

A measurement would be an observation.

测量就是观察。

It's a data point in your database.

它是数据库中的一个数据点。

So the example being power users do four times more bookings, is an all observed fact because your transactional database obviously says that is the case, but it's on an insight because it doesn't have context.

因此，高级用户的预订量增加四倍的例子是一个所有观察到的事实，因为您的事务数据库显然表明情况如此，但它是基于洞察力，因为它没有上下文。

It doesn't give you information that lets you act on it and better understand the problem.

它不会提供让您采取行动并更好地理解问题的信息。

Crystal Widjaja: So another example would be if I see my girlfriend hanging out with a guy I don't know, that is an observed fact that you see in the real world.

Crystal Widjaja：另一个例子是，如果我看到我的女朋友和一个我不认识的人出去玩，这是你在现实世界中看到的一个观察到的事实。

Your hypothesis could be that your girlfriend is cheating on you, but the insight, the actual fact might be that she's not cheating on you, it's her cousin.

你的假设可能是你的女朋友在欺骗你，但洞察力，实际的事实可能是她没有欺骗你，而是她的表弟。

And now your insight is, I am paranoid and I need to change my behavior to be less crazy.

现在你的见解是，我很偏执，我需要改变我的行为，让自己不那么疯狂。

So the insight will provide value when you have this, why answered?

因此，当你有了这个见解时，洞察力就会提供价值，为什么要回答呢？

Why is this person doing this thing?

这个人为什么要做这件事？

Here's why.

原因如下。

And then you are going to act differently.

然后你就会采取不同的行动。

So for our purposes, if we look at a GoFood user will transact and is more likely to use a voucher, that's a fact, that's an observation, but it's not an insight.

因此，就我们的目的而言，如果我们观察 GoFood 用户是否会进行交易并且更有可能使用优惠券，这是事实，这是一种观察，但不是洞察。

An insight would be something like GoFood users who are power users are more likely to use a free shipping discount on a high GMV basket versus non-power users.

一种见解是，与非高级用户相比，高级用户 GoFood 用户更有可能在高 GMV 购物篮中使用免费送货折扣。

Crystal Widjaja: And that actually tells you how to change your marketing approach.

Crystal Widjaja：这实际上告诉你如何改变你的营销方式。

It tells you in what circumstances does someone do this.

它告诉您某人在什么情况下会这样做。

When it's a high GMV basket, give power users the ability to get a free discount, but do not do this for non-powered users because they won't convert any better than they normally would.

当 GMV 较高时，让高级用户能够获得免费折扣，但不要对非高级用户这样做，因为他们的转化不会比平时更好。

So, that helps you change your marketing spend.

因此，这可以帮助您改变营销支出。

It helps you understand the decision points of power users versus non-power users.

它可以帮助您了解高级用户与非高级用户的决策点。

The insight is instrumenting properties into an event so that you can segment who is doing what behavior and make some hypotheses on that observation.

洞察力是将属性插入到事件中，以便您可以细分谁在做什么行为，并根据观察结果做出一些假设。

Test that hypothesis, and then you get some causal representation of whether or not that hypothesis was right.

检验该假设，然后您就可以得到该假设是否正确的一些因果表示。

Lenny: So it sounds like a lot of the root of the issue is setting up the wrong metrics, the wrong...

莱尼：听起来问题的根源在于设置了错误的指标，错误的……

I guess there's the tracking element of just capturing the right information.

我想有一个跟踪元素只是捕获正确的信息。

And then also just not focusing on insights versus just having a bunch of information.

然后，也不关注洞察力，而只关注大量信息。

Lenny: What are signs that you're doing this?

莱尼：有什么迹象表明你正在这样做？

Say someone's going to go load up their dashboard and they're like, "Am I failing or not?" What should they be looking for?

假设有人要加载他们的仪表板，他们会想，“我是否失败了？”他们应该寻找什么？

Crystal Widjaja: So I already know if a team is good at instrumentation or not just by looking at the instrumentation spec.

Crystal Widjaja：所以我已经知道一个团队是否擅长仪器仪表，而不仅仅是通过查看仪表规格。

The symptom of a bad data tracking approach is you have a ton of rows with a ton of events, but every event has one property or no property being tracked.

糟糕的数据跟踪方法的症状是，您有大量包含大量事件的行，但每个事件都有一个属性或没有属性被跟踪。

So an example with Gojek would be when a user lands on the map to select a drop off point, the event would be drop off or map loaded, let's say.

因此，Gojek 的一个例子是，当用户登陆地图选择下车点时，事件将被下车或地图加载。

And the properties there should be things like how many drivers do they see on the screen?

那里的属性应该是诸如他们在屏幕上看到多少个驱动程序之类的东西？

What is the pickup location?

上车地点是什么？

What city is it in?

它在哪个城市？

What latitude and longitude is it?

它的纬度和经度是多少？

Is there surge pricing?

有峰时定价吗？

What is the current minimum fare?

目前的最低票价是多少？

Do they have a voucher code?

他们有优惠券代码吗？

Crystal Widjaja: All of these characteristics of the experience and the context that can help you look at hey, when a user only sees two drivers on the screen, they're much less likely to convert than a user who sees five drivers on a screen.

Crystal Widjaja：体验和上下文的所有这些特征都可以帮助您了解，嘿，当用户在屏幕上只看到两个驱动程序时，他们转换的可能性比在屏幕上看到五个驱动程序的用户要小得多。

Now we can look at in what cities and in what latitude and longitudes do we mostly only see two drivers versus five drivers.

现在我们可以看看在哪些城市以及哪些纬度和经度，我们大多只看到两名司机而不是五名司机。

Being able to do the second layer approach of the why and not just stop at, "That's weird.

能够采用第二层方法来解释原因，而不仅仅是停留在“这很奇怪。

When you have two drivers you are less likely to book." But then you never ask why.

当你有两名司机时，你预订的可能性就较小。”但你永远不会问为什么。

That drives me crazy.

这让我发疯。

Or the inability to even know that there were only two drivers on the screen.

或者甚至无法知道屏幕上只有两个驱动程序。

You're missing so much context of the user's experience that you're unable to make assumptions about why the user didn't convert.

您缺少太多有关用户体验的背景信息，以至于您无法假设用户没有进行转化的原因。

Lenny: I love this.

莱尼：我喜欢这个。

Maybe your course is probably going to be the answer, but for folks that want to figure out how to do this taxonomy and events well, how do they go about doing that?

也许你的课程可能就是答案，但是对于那些想要弄清楚如何做好这种分类和事件的人来说，他们该如何去做呢？

Crystal Widjaja: So I think it's important to just go through examples.

Crystal Widjaja：所以我认为仅通过示例很重要。

Yes, every product is different, but everyone has the same signup flow for the most part.

是的，每种产品都是不同的，但大多数情况下每个人都有相同的注册流程。

So look at the signup flow examples that I have in the blog post or in, I believe Amplitude actually has a pretty good long-winded documentation on this, on how to do a event tracking.

因此，看看我在博客文章或中的注册流程示例，我相信 Amplitude 实际上有一个关于如何进行事件跟踪的非常好的冗长文档。

But it's really a matter of sitting down and thinking really deeply.

但这确实是一个坐下来认真思考的问题。

If I were to press this button, why would I and why would I not?

如果我按下这个按钮，我为什么会按下，为什么不按下？

And am I tracking that in my user properties?

我是否在我的用户属性中跟踪它？

So it's really just sitting down and mapping out the experience.

所以实际上只是坐下来规划体验。

Lenny: Speaking of Amplitude and other data tools, do you have a default recommended metrics stack for our founders just to start with and maybe a few other things as they evolve?

Lenny：说到 Amplitude 和其他数据工具，您是否有一个默认的推荐指标堆栈供我们的创始人开始使用，也许随着他们的发展还有其他一些东西？

Crystal Widjaja: That really depends on how early they are.

Crystal Widjaja：这实际上取决于他们的早到程度。

So if they have a single data warehouse with all of their transactional data, usually I say, you can probably get by with Google Data Studio.

因此，如果他们有一个包含所有交易数据的数据仓库，通常我会说，你可能可以使用 Google Data Studio。

It's free usually with whatever you're using.

无论您使用什么，它通常都是免费的。

If not Metabase has a great open source free tool.

如果没有的话，Metabase 有一个很棒的开源免费工具。

If you have someone who can write SQL or if you have multiple databases, then Metabase is great.

如果您有人会编写 SQL 或者您有多个数据库，那么 Metabase 就很棒。

If you need in app mobile device event tracking, I usually recommend CleverTap because Mixpanel has unfortunately failed me a lot.

如果您需要在应用程序中跟踪移动设备事件，我通常推荐 CleverTap，因为 Mixpanel 不幸地让我失望了很多。

And Amplitude doesn't have the CRM components that I would need all in one space.

而且 Amplitude 没有将我需要的所有 CRM 组件集中在一个空间中。

Crystal Widjaja: If I am much bigger and I need more analytics juice, maybe Amplitude makes sense on top of this, or something that helps me pipe data into more dashboards and do less ETL for me.

Crystal Widjaja：如果我的规模更大并且需要更多的分析能力，也许 Amplitude 更有意义，或者可以帮助我将数据传输到更多仪表板并为我执行更少的 ETL。

Then I would get into Segment.

然后我会进入细分市场。

And then once you get into experimentation, obviously I have to shout out to Eppo.

一旦你开始实验，显然我必须向 Eppo 大声喊叫。

I think they've really instrumented a lot of the dashboards that I would've normally had to do in experimentation projects.

我认为他们确实对我通常在实验项目中必须做的很多仪表板进行了检测。

So I usually look at something like Eppo to just automate the decision making flow.

因此，我通常会使用 Eppo 之类的工具来自动化决策流程。

Lenny: Awesome.

莱尼：太棒了。

I think we're both small investors in Eppo, big fans, a little bit of bias, but yeah's it's an excellent Airbnb team that built it, so it's cool.

我认为我们都是 Eppo 的小投资者，大粉丝，有点偏见，但是是的，这是一个优秀的 Airbnb 团队建造的，所以它很酷。

Shifting a bit from metrics and data to just growth teams in general, maybe first question is just, how do you recommend companies set up a growth team in the early days and then over time?

从指标和数据转向一般的增长团队，也许第一个问题只是，您如何建议公司在早期建立增长团队，然后随着时间的推移？

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

So I can talk about how growth was set up at Gojek as an example, which I think is probably the best practice.

所以我可以以 Gojek 为例来谈谈增长是如何建立的，我认为这可能是最佳实践。

So we didn't really know what growth was at that time, but we knew there were obvious gaps to fill.

所以当时我们并不真正知道增长是什么，但我们知道有明显的空白需要填补。

So because we had grown so quickly, the core product team was still making the core product features.

因此，由于我们发展得如此之快，核心产品团队仍在开发核心产品功能。

As simple as phone number masking.

就像电话号码屏蔽一样简单。

That wasn't a thing yet.

那还不是事。

You had access to your driver's phone number.

您可以访问司机的电话号码。

It's probably not a great thing.

这或许不是什么好事。

It's probably part of the core functionality and we need to fill that gap.

它可能是核心功能的一部分，我们需要填补这个空白。

At the same time, growth was still necessary because you had all of these users trying to use a product that aren't quite getting there.

与此同时，增长仍然是必要的，因为所有这些用户都在尝试使用尚未完全实现目标的产品。

Crystal Widjaja: So things like figuring out what SMS provider we should use to send the OTP to this user who is signing up from this telco provider.

Crystal Widjaja：比如弄清楚我们应该使用哪个 SMS 提供商将 OTP 发送给从该电信提供商注册的用户。

That was a growth objective that isn't necessarily core feature work, but was a gap to fill given the onboarding and SMS success delivery rates.

这是一个增长目标，不一定是核心功能工作，但考虑到入职和短信成功发送率，这是一个需要填补的空白。

Things like telling the driver if this was a brand new customer, because at this point in time, drivers had taken thousands of rides and they assumed every single customer knew how Gojek worked, when maybe they didn't.

比如告诉司机这是否是一个全新的客户，因为在这个时候，司机已经乘坐了数千次车，他们认为每个客户都知道 Gojek 是如何工作的，而实际上他们可能并不知道。

And so we knew that the protocol was that a power user would know they would make an order and they would just wait.

所以我们知道协议是高级用户知道他们会下订单并且他们只会等待。

They would wait somewhere, they would keep an eye out for a driver and then they would get on the motorcycle and go.

他们会在某个地方等待，留意司机，然后骑上摩托车出发。

Crystal Widjaja: But for a brand new user, are you supposed to walk to the driver?

Crystal Widjaja：但是对于一个全新的用户来说，你应该走到司机那里吗？

Are you supposed to find them?

你应该找到他们吗？

It's unclear to this brand new, uneducated new user how to use the product.

对于这个全新的、未受过教育的新用户来说，不清楚如何使用该产品。

And so first time user experience could have been a terrible one where they went and walked off and then the driver came to the pickup point and they couldn't find them.

因此，第一次用户体验可能会很糟糕，他们走开，然后司机来到接送点，但他们找不到他们。

So it was all of these small acquisition, adoption and engagement use cases that growth was filling the gap on.

因此，正是所有这些小型收购、采用和参与用例，增长才填补了空白。

And eventually we embedded our growth, I would say product managers at the time, into these teams and they ended up synthesizing what growth was as a full-time role.

最终，我们将我们的增长（我想说当时的产品经理）融入到这些团队中，他们最终将增长作为一个全职角色进行了综合。

Eventually becoming PMs who own specific parts of the product stack.

最终成为拥有产品堆栈特定部分的产品经理。

Lenny: So in your experience, and I hear this a lot, is your first growth person shouldn't just come in and figure out what to work on.

莱尼：所以根据你的经验，我经常听到这样的话，你的第一个成长型员工不应该只是进来弄清楚要做什么。

You should understand here's where we need growth help, let's find somebody to tackle it, versus come help us figure out what to do to drive growth.

你应该明白，这就是我们需要增长帮助的地方，让我们找人来解决这个问题，而不是来帮助我们弄清楚如何推动增长。

Is that how you've seen it?

你是这样看的吗？

Crystal Widjaja: Exactly.

克里斯特尔·维贾贾：确实如此。

I think it's just setting the bar too high to expect someone to come in and model everything.

我认为这只是把标准设置得太高了，以至于不能指望有人进来对一切进行建模。

Again, there are physics in place it's very hard to move everything.

再说一遍，有物理原理，很难移动所有东西。

So it's really about having someone who already has all of this data knows where the biggest gaps are.

因此，真正重要的是让已经拥有所有这些数据的人知道最大的差距在哪里。

Doesn't have to start from scratch and figure this out and then just picks some small space to work on that they know is workable.

不必从头开始解决这个问题，然后只选择一些他们知道可行的小空间来工作。

Lenny: Do you have strong opinions about growth being integrated?

Lenny：您对于整合增长有强烈的看法吗？

The way that you described where growth PM basically has a cross functional team basically is the PM versus a separate growth team that's off to the side.

您所描述的增长 PM 基本上拥有一个跨职能团队的方式基本上是 PM 与偏向一边的独立增长团队。

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

I think it can work as a separate growth team to the side if the company is truly head over heels, tripping on insane product market fit, if there's insane, product market fit and you are really scrambling to do core feature stacks, then maybe a growth team to come and be clean up is fine.

我认为，如果公司真的神魂颠倒，被疯狂的产品市场契合度所绊倒，那么它可以作为一个独立的增长团队在一边工作，如果有疯狂的产品市场契合度，而你真的在争先恐后地做核心功能堆栈，那么也许一个增长团队来清理就可以了。

We're the cleanup crew.

我们是清洁工。

We pick up the pieces that were left behind, we connect the dots.

我们拾起留下的碎片，将这些点连接起来。

You forgot to plug this in, we'll plug it in for you.

您忘记插入此插头，我们会为您插入。

But we were a team of lots of stats heavy people.

但我们是一支由很多注重数据的人组成的团队。

So a lot of my team were statistics graduates.

所以我的团队中有很多人都是统计学毕业生。

We cared a lot about looking at numbers and odds and probabilities because it really is a numbers game at that scale.

我们非常关心数字、赔率和概率，因为这确实是一个如此规模的数字游戏。

You could work on anything and everything would probably do something.

你可以做任何事，而且每件事都可能有所作为。

But what was the thing that would make the most impact now and unlock us for the future.

但什么是对现在产生最大影响并为我们开启未来的东西呢？

Lenny: I was going to ask you folks to look for when they're hiring an early growth person, is that what you find, just stats, data kind of person?

莱尼：我想请你们在招聘早期成长型人才时寻找这样的人，你们找到的只是统计数据类型的人吗？

Crystal Widjaja: You have to have someone who knows how to run the numbers.

Crystal Widjaja：你必须有人知道如何计算数字。

If you're looking at ratios of conversion rates, but you don't realize that this ratio is of a much smaller base size, you're going to make the wrong decision.

如果您正在查看转化率比率，但您没有意识到该比率的基础规模要小得多，那么您将做出错误的决定。

So someone who is intuitively good at statistics, they know how to do sampling appropriately.

因此，直觉上擅长统计的人，他们知道如何适当地进行抽样。

They know what selection bias is.

他们知道什么是选择偏差。

The worst possible thing is to have a growth person who thinks they are doing the right thing and is measuring things wrong and then focusing on the wrong areas.

最糟糕的事情是，成长型人才认为自己正在做正确的事情，但却错误地衡量了事情，然后将注意力集中在错误的领域。

Lenny: Do you find that it's often easier or better to hire a young up and coming person or find someone that's got a bunch of experience for your first growth hire?

莱尼：您是否发现，雇用一位年轻有为的人才或找到一位经验丰富的人作为您的第一次成长招聘通常更容易或更好？

Crystal Widjaja: I would hire someone who is willing to take intro to statistics course.

Crystal Widjaja：我会聘请愿意参加统计课程入门的人。

And it doesn't matter if they've had the experience to go wild or not.

他们是否有过疯狂的经历并不重要。

I think it really is, can they focus on the right opportunity rather than the most flashy thing?

我想确实如此，他们能否专注于正确的机会而不是最华而不实的事情？

And I think both profiles can come under that.

我认为这两种情况都可以归入这一范畴。

Lenny: Got it.

莱尼：明白了。

And then what do you do in a hiring process for someone like this?

那么在招聘过程中你会为这样的人做什么呢？

What kind of things do you suggest founders look for?

您建议创始人寻找什么样的东西？

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

I actually look for that first principle bias.

我实际上在寻找第一原则偏见。

So I'll give people case studies of here's what we see, how do you know that this is true?

因此，我将向人们提供我们所看到的案例研究，你怎么知道这是真的？

And then I have them set up an experiment design.

然后我让他们设置了一个实验设计。

I want to see that they are sampling randomly.

我想看到他们是随机抽样的。

Not that they're like, "I'm going to build this feature and launch it, and of course it's going to work." I want to see that they're taking a measured deliberate approach to considering why someone might do this or what tools are available.

他们并不是说，“我要构建这个功能并推出它，它当然会起作用。”我希望看到他们采取审慎的方法来考虑为什么有人会这样做或者有哪些工具可用。

A growth team can go terribly wrong when they just try to onboard a bunch of brand new tools that don't integrate well and it takes six months to integrate fully, and then they get nothing done for six months.

当增长团队只是尝试使用一堆无法很好集成的全新工具并且需要六个月才能完全集成，然后他们在六个月内什么也没做时，他们可能会犯下严重错误。

Everything in growth is an opportunity cost of time, trade off with what you could have been doing to the product in that time.

增长中的一切都是时间的机会成本，权衡你当时可以对产品做的事情。

Crystal Widjaja: So we biased towards really quick hacky things.

Crystal Widjaja：所以我们偏向于真正快速的黑客事物。

Like in the early days of Gojek growth, I think our first real growth experiment, we were actually still the data team at this time, was to connect a quick Python script to the Twilio API that we had access to.

就像 Gojek 发展的早期一样，我认为我们的第一个真正的增长实验（当时我们实际上仍然是数据团队）是将一个快速的 Python 脚本连接到我们可以访问的 Twilio API。

And we SMSd a bunch of drivers through a CSV that we uploaded that said like, "Hey, your acceptance rate is really low.

我们通过上传的 CSV 向一群司机发送短信，内容如下：“嘿，你们的接受率真的很低。

You're not supposed to do that.

你不应该这样做。

Please accept all the rides that you are getting." And that actually increased acceptance rates by 2% across the board.

请接受您所获得的所有游乐设施。”这实际上使整体录取率提高了 2%。

And when we looked deeper into that data, it did even more so for brand new drivers.

当我们更深入地研究这些数据时，对于全新的驾驶员来说更是如此。

And so we then worked with the data driver onboarding team so that they could better facilitate the onboarding experience for their drivers.

因此，我们随后与数据驾驶员入职团队合作，以便他们能够更好地促进驾驶员的入职体验。

Lenny: For the interview question that you described, an experiment design question, do you give that as a project where they have time to work on it or is it a live thing?

莱尼：对于你所描述的面试问题，一个实验设计问题，你是否将其作为一个项目，让他们有时间进行研究，或者它是一个实时的事情？

Crystal Widjaja: Yes.

克里斯特尔·维贾贾：是的。

Yeah.

是的。

I don't think live works really well for these case studies.

我认为现场直播对于这些案例研究来说效果并不好。

I want to see people put in the time and the work to do something to the best of their ability.

我希望看到人们投入时间和工作，尽其所能地做一些事情。

And of course we ask them like, "Hey, you have five days.

当然，我们会问他们，“嘿，你有五天时间。

We expect you to spend probably four hours on this, so if you don't have four hours within these five days, let us know." So we're pretty careful about giving them the appropriate amount of time to do it at the level of quality that we would've expected if they were to work here full-time.

我们预计您可能会花四个小时来完成此任务，因此，如果您在这五天内没有四个小时，请告诉我们。”因此，我们非常谨慎地给予他们适当的时间，以达到我们在他们全职工作时所期望的质量水平。

So give them those four hours, we want to see do they Google.

所以给他们这四个小时，我们想看看他们是否谷歌。

If they can't figure it out right now, let's see them Google it.

如果他们现在无法弄清楚，让我们看看他们用谷歌搜索一下。

We'll ask them what approaches they took, how did they figure this out.

我们会问他们采取了什么方法，他们是如何解决这个问题的。

And we like to hear people say that they literally had to Google this and read a bunch of white papers.

我们喜欢听到人们说他们确实必须用谷歌搜索并阅读一堆白皮书。

I do that as well.

我也这样做。

Lenny: For people trying to design one of these for themselves, do you have a question that you've retired that you could share or something that would help somebody design their own prompt?

莱尼：对于试图为自己设计其中一个提示的人来说，您是否有一个已经退休的问题可以分享，或者有什么可以帮助别人设计自己的提示的问题？

Crystal Widjaja: Yeah.

克里斯特尔·维贾贾：是的。

I can give you a template after this call.

这次通话后我可以给你一个模板。

Lenny: Amazing.

莱尼：太棒了。

We'll include that in the show notes.

我们会将其包含在演出笔记中。

Easy peasy.

简单易行。

Amazing.

惊人的。

Okay.

好的。

A last topic that I wanted to cover is a very cool thing that you were involved in.

我想讨论的最后一个主题是您参与的一件非常酷的事情。

It's a nonprofit that you started called Generation Girl.

这是一个由您创办的非营利组织，名为“Generation Girl”。

And I think the mission is to help women and young girls get into STEM.

我认为我们的使命是帮助女性和年轻女孩进入 STEM。

So I'd love to hear about this program, how you got into it, what it's all about, and then also just how listeners can help support what you're doing.

所以我很想听听这个计划，你是如何参与这个计划的，它是关于什么的，以及听众如何帮助支持你正在做的事情。

Crystal Widjaja: Absolutely.

克里斯特尔·维贾贾：当然。

Yes.

是的。

Generation Girl is very near and dear to my heart.

《一代女孩》对我来说非常亲近。

So I co-founded this with a couple of amazing women who were also at Gojek, but are now full-time at Generation Girl.

因此，我与几位出色的女性共同创立了这个公司，她们也曾在 Gojek，但现在全职在 Generation Girl。

So this really stemmed from us repeatedly getting annoying comments about working in STEM.

所以这实际上是因为我们不断收到关于 STEM 工作的恼人评论。

So things like, "You can't possibly be the engineer on this project.

比如，“你不可能是这个项目的工程师。

You look like you like makeup and stuff." And we were like, "Yes, I absolutely love makeup, but I also am badass at writing SWIFT code, so step aside." So having experienced a lot of the misrepresentation of what an engineer should look like or should like, I think we really look to Legally Blonde, is one of my favorite movies that represents you can take the powers that you have, whether you like engineering or design or data, and you can be whoever you want and still kick ass at it.

你看起来很喜欢化妆之类的东西。”我们的反应是，“是的，我非常喜欢化妆，但我也很擅长编写 SWIFT 代码，所以靠边站吧。”因此，在经历了很多关于工程师应该是什么样子或应该喜欢什么的误传之后，我认为我们真的很喜欢《律政俏佳人》，这是我最喜欢的电影之一，它代表着你可以掌握你所拥有的权力，无论你喜欢工程、设计还是数据，你可以成为任何你想要的人，并且仍然可以尽情发挥。

Crystal Widjaja: So a lot of the women that we support, we're actually happy if they go into one of our classes and they say, "Actually, I don't like engineering." That's great.

Crystal Widjaja：我们支持的很多女性，如果她们进入我们的一门课程并说：“实际上，我不喜欢工程。”我们实际上很高兴。那太棒了。

That's agency and empowerment that they got to make that decision for themselves without any cultural biases or social pressure telling them that they should feel this way.

这是他们必须自己做出决定的代理和授权，没有任何文化偏见或社会压力告诉他们应该有这种感觉。

And so we offer free classes for girls 12 to 17.

因此，我们为 12 至 17 岁的女孩提供免费课程。

We have college classes.

我们有大学课程。

We partner with teachers about how to teach STEM topics, especially in areas where they don't have laptops for every student.

我们与教师合作，探讨如何教授 STEM 主题，特别是在他们没有为每个学生配备笔记本电脑的领域。

How do you teach how to use Figma and things like that?

你是如何教授如何使用 Figma 之类的东西的？

So people can definitely support us and reach out to us.

所以人们绝对可以支持我们并与我们联系。

We have a PayPal on our website, take a look.

我们的网站上有 PayPal，您可以看看。

Lenny: Can you share some of the impact that you've seen from this?

莱尼：您能分享一下您从中看到的一些影响吗？

Are there numbers you can share or anything that you can share around what the organizations have done.

您是否可以分享有关组织所做工作的数字或任何内容？

Crystal Widjaja: So we've already had several thousand students go through Generation Girl, summer clubs and programs and classes.

Crystal Widjaja：我们已经有数千名学生参加了“Generation Girl”、暑期俱乐部、项目和课程。

So we have an event every week.

所以我们每周都会举办一次活动。

We have a full summer club that's every single day for two weeks, every summer and every winter.

我们有一个完整的夏季俱乐部，每年夏天和每年冬天，两周内每天都举行。

We have partnerships with some of the biggest tech companies in Indonesia, where we partner students with engineers and they work on projects together.

我们与印度尼西亚一些最大的科技公司建立了合作伙伴关系，我们将学生与工程师合作，他们一起开发项目。

And most recently we're part of the MIT solve program with our new initiative Class.

最近，我们通过新的倡议类参与了麻省理工学院解决方案项目。

So Class, we're creating a free to use site for teachers.

所以，班级，我们正在为教师创建一个免费使用的网站。

Crystal Widjaja: So right now we have partnered with a handful of universities in Indonesia, both in rural and city of Jakarta where teachers can now have the knowledge and material to explain newer concepts that maybe they're less familiar with, because startup world changes rapidly, how you develop changes rapidly.

Crystal Widjaja：所以现在我们已经与印度尼西亚的几所大学合作，无论是在雅加达的农村还是城市，教师现在可以获得知识和材料来解释他们可能不太熟悉的新概念，因为创业世界瞬息万变，你的发展方式也迅速变化。

So this is one thing that we are most excited about because every teacher impacts thousands of students a year.

所以这是我们最兴奋的一件事，因为每位老师每年都会影响成千上万的学生。

And being able to teach the teachers and give them the resources that they need is something that's really important.

能够教授教师并为他们提供所需的资源是非常重要的。

Lenny: That's incredible.

莱尼：这太不可思议了。

It's currently just in Southeast Asia, is that right?

目前仅在东南亚，对吗？

Crystal Widjaja: Only in Indonesia, because frankly, this is where everyone needs the most support.

Crystal Widjaja：只有在印度尼西亚，因为坦白说，这是每个人最需要支持的地方。

Globally STEM is not well received or welcoming at all to women.

在全球范围内，STEM 并没有受到女性的欢迎或欢迎。

I think it's gotten worse over the past few decades.

我认为过去几十年来情况变得更糟。

Below 18% of college graduates are women in computer science.

计算机科学专业的女性大学毕业生比例低于 18%。

So we're really trying to reach the youngest generation because that's when you are told or informed that computer science is for specific types of people.

因此，我们确实在努力接触最年轻的一代，因为那时您会被告知或被告知计算机科学是针对特定类型的人的。

Lenny: It's really sad to hear that it's heading in the wrong direction.

莱尼：听到它正朝着错误的方向前进，我真的很难过。

What do you think is contributing to that?

您认为是什么促成了这一点？

Crystal Widjaja: I think there is still a lot of this mental model of what a computer scientist is able to do and how much support they're given.

Crystal Widjaja：我认为仍然存在很多关于计算机科学家能够做什么以及他们获得多少支持的心理模型。

So it's been shown in studies that at the youngest generation middle school, high school, you are more likely to be given introductory STEM classes as a male than as a female.

研究表明，在最年轻的一代初中、高中，男性比女性更有可能接受 STEM 入门课程。

So women just aren't targeted for STEM at that younger age.

因此，女性在那么年轻的时候并不是 STEM 的目标。

And so when they enter the high school or college classes for computer science, they're way behind.

因此，当他们进入高中或大学计算机科学课程时，他们就远远落后了。

And that does not feel good.

这感觉不太好。

No one likes to be the worst in the class.

没有人喜欢成为班上最差的。

And so it's more likely that you'll drop out.

所以你更有可能辍学。

We've seen studies at Carnegie Mellon that actually would create introductory computer science classes before the college class starts.

我们已经看到卡内基梅隆大学的研究实际上会在大学课程开始之前开设计算机科学入门课程。

And for the women who did join those classes, they actually graduated at similar rates as their male counterparts.

对于参加这些课程的女性来说，她们的毕业率实际上与男性同行相似。

So it's really setting them up for success.

所以这确实为他们的成功奠定了基础。

Lenny: If folks want to help.

莱尼：如果大家愿意帮忙的话。

You said that there's a PayPal page.

你说有一个 PayPal 页面。

Is there any other sort of action people can take?

人们还可以采取其他行动吗？

Crystal Widjaja: Yes.

克里斯特尔·维贾贾：是的。

Enterprise software.

企业软件。

We love to teach iOS development, licensed software.

我们喜欢教授 iOS 开发、许可软件。

We have hundreds of students a year, so let us know.

我们每年有数百名学生，所以请告诉我们。

Lenny: Awesome.

莱尼：太棒了。

And they can reach you on generationgirl.com?

他们可以通过 Generationgirl.com 联系到您吗？

Lenny: Crystal, thank you so much for being here.

莱尼：克里斯托，非常感谢你来到这里。

I've taken enough of your time.

我已经占用你足够多的时间了。

Two last quick questions.

最后两个问题。

Where can folks find you online if they want to reach out?

如果人们想联系您，可以在哪里找到您？

And then other than the Generation Girl chat we just had, is there any other way folks can be helpful to you?

那么除了我们刚刚进行的“Generation Girl”聊天之外，人们还有其他方式可以帮助您吗？

Crystal Widjaja: Yes.

克里斯特尔·维贾贾：是的。

Please find me at crystalwidjaja.com.

请在 crystalwidjaja.com 上找到我。

You can reach out to me and my email is there.

您可以联系我，我的电子邮件就在那里。

Listeners, please do instrumentation correctly.

各位听众，请正确地演奏乐器。

Please don't track your KPIs.

请不要跟踪您的 KPI。

Please track your user journeys and experiences.

请跟踪您的用户旅程和体验。

We'll have much funner things to talk about if you do that.

如果你这样做的话，我们会有很多有趣的事情可以谈论。

Lenny: Amazing PSA.

莱尼：惊人的公益广告。

Thank you so much, Crystal.

非常感谢你，水晶。

Crystal Widjaja: Thanks Lenny.

Crystal Widjaja：谢谢莱尼。

This was a blast.

这真是太棒了。

Lenny: Thank you so much for listening.

莱尼：非常感谢您的聆听。

If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify or your favorite podcast app.

如果您发现这很有价值，您可以在 Apple Podcasts、Spotify 或您最喜欢的播客应用程序上订阅该节目。

Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast.

另外，请考虑给我们评分或留下评论，因为这确实有助于其他听众找到播客。

You can find all past episodes or learn more about the show at lennyspodcast.com.

您可以在 lennyspodcast.com 上找到所有过去的剧集或了解有关该节目的更多信息。

See you in the next episode.

下一集见。