The Internet's hidden science factory
Marshall is a worker for Amazon’s Mechanical Turk, an online job forum where “requesters” post jobs, and an army of crowdsourced workers complete them, earning fantastically small fees for each task.
马歇尔是亚马逊 Mechanical Turk 的一名工作人员，这是一个在线求职论坛，“申请者”在这里发布工作，然后由一群众包工作人员完成这些工作，每项任务赚取的费用极少。
The work has been called microlabor, and the jobs, known as Human Intelligence Tasks, or HITs, range wildly.
这项工作被称为“微劳动”，而这些工作被称为“人类智能任务”（HIT），范围广泛。
Some are tedious: transcribing interviews or cropping photos.
有些是乏味的：抄写采访或裁剪照片。
Some are funny: prank calling someone’s buddy (that’s worth $1) or writing the title to a pornographic movie based on a collection of dirty screen grabs (6 cents).
有些很有趣：恶作剧地打电话给某人的好友（价值 1 美元），或者根据一系列肮脏的屏幕抓取内容为色情电影编写标题（6 美分）。
And others are downright bizarre.
还有一些则非常奇怪。
One task, for example, asked workers to strap live fish to their chests and upload the photos.
例如，一项任务要求工作人员将活鱼绑在胸前并上传照片。
That paid $5 — a lot by Mechanical Turk standards.
这笔费用为 5 美元——按照 Mechanical Turk 的标准来说，这已经是一笔巨款了。
Mostly, Marshall is a sort of cyber guinea pig, providing a steady stream of data to academic and scientific research.
大多数情况下，马歇尔是一种网络豚鼠，为学术和科学研究提供源源不断的数据。
This places her squarely inside a growing culture of super-savvy, highly experienced study participants.
这使她完全置身于一个由超级精明、经验丰富的研究参与者组成的不断发展的文化之中。
As she works, she hears a rustling noise.
当她工作时，她听到沙沙的声音。
“Grayson, are you in my garbage can?”
“格雷森，你在我的垃圾桶里吗？”
In the kitchen, the trash can’s on its side.
在厨房里，垃圾桶放在一边。
Her son has liberated an empty box of cinnamon rolls and dumped the remaining contents on the floor.
她的儿子拿出一个空盒子肉桂卷，把剩下的东西倒在地板上。
She goes to him, scoops him up and carries him back to the living room, where he circles the carpet, chattering happily as she resumes typing.
她走到他身边，把他抱起来，带回客厅，他在地毯上转圈，当她继续打字时，他高兴地喋喋不休。
“I’m never going to be absolutely undistracted, ever,” Marshall says, and smiles.
“我永远不会完全不分心，永远，”马歇尔微笑着说道。
Her employers don’t know that Marshall works while negotiating her toddler’s milk bottles and giving him hugs.
她的雇主不知道马歇尔一边工作一边处理她孩子的奶瓶并拥抱他。
They don’t know that she has seen studies similar to theirs maybe hundreds, possibly thousands, of times.
他们不知道她已经看过类似他们的研究数百次，甚至数千次。
“It’s hard to reproduce a gut response when you’ve answered a survey that’s basically the same 200 times.” Since its founding in 2005, Mechanical Turk has become an increasingly popular way for university researchers to recruit subjects for online experiments.
“当你回答了 200 次基本上相同的调查时，很难重现直觉反应。”自 2005 年成立以来，Mechanical Turk 已成为大学研究人员招募在线实验受试者的一种越来越流行的方式。
It’s cheap, easy to use, and the responses, powered by the forum’s 500,000 or so workers, flood in fast.
它价格便宜、易于使用，并且由论坛约 500,000 名工作人员提供的响应迅速涌入。
These factors are such a draw for researchers that, in certain academic fields, crowdsourced workers are outpacing psychology students — the traditional go-to study subjects.
这些因素对研究人员来说非常有吸引力，以至于在某些学术领域，众包工作者的发展速度超过了心理学学生——传统的首选学习科目。
And the studies are a huge draw for many workers, who tend to participate again and again and again.
这些研究对许多工人来说具有巨大的吸引力，他们往往会一次又一次地参与。
These aren’t obscure studies that Turkers are feeding.
这些并不是土耳其人正在进行的晦涩的研究。
They span dozens of fields of research, including social, cognitive and clinical psychology, economics, political science and medicine.
他们跨越数十个研究领域，包括社会、认知和临床心理学、经济学、政治学和医学。
They teach us about human behavior.
他们教给我们关于人类行为的知识。
They deal in subjects like energy conservation, adolescent alcohol use, managing money and developing effective teaching methods.
他们研究的主题包括节能、青少年饮酒、资金管理和开发有效的教学方法。
“Most of what’s happening in these studies involves trying to understand human behavior,” said Yale University’s David Rand.
“这些研究中发生的大部分事情都涉及试图理解人类行为，”耶鲁大学的大卫·兰德说。
“Understanding bias and prejudice, and how you make financial decisions, and how you make decisions generally that involve taking risks, that kind of thing.
“了解偏见和成见，以及如何做出财务决策，以及如何做出通常涉及冒险的决策，诸如此类的事情。
And there are often very clear policy implications.”
而且往往会产生非常明确的政策影响。”
As the use of online crowdsourcing in research continues to grow, some are asking the question: How reliable are the data that these modern-day research subjects generate?
随着在线众包在研究中的使用不断增长，一些人提出了这样的问题：这些现代研究课题生成的数据有多可靠？
In 2010, the researcher Joseph Henrich and his team published a paper showing that an American undergraduate was about 4,000 times more likely than an average American to be the subject of a research study.
2010年，研究员约瑟夫·亨里奇（Joseph Henrich）和他的团队发表了一篇论文，表明美国本科生成为研究对象的可能性比普通美国人高出约4000倍。
But that output pales in comparison to Mechanical Turk workers.
但与 Mechanical Turk 工人相比，这一产量就显得相形见绌了。
The typical “Turker” completes more studies in a week than the typical undergraduate completes in a lifetime.
典型的“Turker”一周完成的学业比典型的本科生一生完成的学业还要多。
That’s according to research by Rand, who surveyed both groups.
这是根据兰德的研究得出的，他对这两个群体进行了调查。
Among those he surveyed, he found that the median traditional lab subject had completed 15 total academic studies — an average of one per week.
他发现，在他调查的对象中，传统实验室受试者的中位数已完成 15 项学术研究，平均每周完成一项。
The median Turker , on the other hand, had completed 300 total academic studies — an average of 20 per week.
另一方面，Turker 的中位数完成了 300 项学术研究——平均每周 20 项。
“Which is just crazy,” Rand said.
“这太疯狂了，”兰德说。
“And for a lot of experiments, that’s a big problem.”
“对于很多实验来说，这是一个大问题。”
David Rand, director of Yale University’s Human Cooperation Laboratory, presents at the PopTech Conference in Camden, Maine in 2012.
耶鲁大学人类合作实验室主任 David Rand 于 2012 年在缅因州卡姆登举行的 PopTech 会议上发表演讲。
Photo by Thatcher Cook
撒切尔·库克摄
Rand, a young, energetic behavioral economist, who accessorizes his suit jacket with gray converse shoes and orange-striped socks, works on the second floor of a beautiful cathedral-like building on Yale’s main campus.
兰德是一位年轻、充满活力的行为经济学家，他在耶鲁大学主校区一座美丽的大教堂式建筑的二楼工作，他的西装外套搭配灰色匡威鞋和橙色条纹袜子。
Behind his desk are shelves of robot toys, a nod to his pre-professor days, when he fronted an electro-punk band called Robot Goes Here.
他办公桌后面的架子上摆满了机器人玩具，这是对他还是教授之前的日子的致敬，当时他领导了一支名为“Robot Goes Here”的电子朋克乐队。
“I actually had a record deal,” he said.
“我实际上签了唱片合约，”他说。
Rand was an early proselytizer for Mechanical Turk.
兰德是机械土耳其人的早期劝说者。
In fact, he authored the first study in his field that encouraged scientists to tap Turkers for surveys.
事实上，他撰写了该领域的第一项研究，鼓励科学家利用土耳其人进行调查。
At the time, he gave talks to fellow researchers , telling them recruiting via Mechanical Turk could be done more quickly, cheaply and easily and could be “just as valid as other types of experiments.”
当时，他与其他研究人员进行了交谈，告诉他们通过 Mechanical Turk 进行招募可以更快、更便宜、更容易，并且“与其他类型的实验一样有效”。
That was in 2010.
那是在2010年。
Since then, his early enthusiasm has been tempered with caution.
从那时起，他早期的热情就被谨慎地调和了。
He’s been following the forum for nearly a decade and has come to believe that it has some serious limitations.
他关注该论坛近十年了，并逐渐相信该论坛存在一些严重的局限性。
First, there’s the question of dropout rates.
首先，存在辍学率问题。
Turkers are more likely to drop out mid-study, and that can skew the results.
土耳其人更有可能在学习中途退学，这可能会影响结果。
Then there’s the question of environmental control.
然后是环境控制的问题。
Read: There is none.
读：没有。
In the lab, it’s easy to monitor survey takers; not so online.
在实验室中，很容易监控调查对象；不那么在线。
Who’s to say they’re not watching reality television while working, or drinking a few beers on the job?
谁敢说他们不在工作时看电视真人秀，或者在工作时喝几杯啤酒？
To guard against this, researchers test a worker’s focus by planting “attention checks” in their surveys.
为了防止这种情况发生，研究人员通过在调查中植入“注意力检查”来测试员工的注意力。
“Have you ever eaten a sandwich on Mars,” a question might read.
“你在火星上吃过三明治吗？”一个问题可能会这样写。
Or “Have you ever had a fatal heart attack?” But the attention check questions are often recycled, and experienced workers spot them immediately.
或者“你有过致命的心脏病发作吗？”但注意力检查问题经常被重复使用，经验丰富的工作人员会立即发现它们。
(“Whenever I see the word vacuum, I know it’s an attention check,” Marshall has said.)
（“每当我看到真空这个词时，我就知道这是一次注意力检查，”马歇尔说。）
But it’s the absence of gut intuition from experienced workers that concerns Rand the most.
但兰德最担心的是经验丰富的员工缺乏直觉。
A person’s gut response to a question is an important measurement in many social psychology studies.
在许多社会心理学研究中，一个人对问题的本能反应是一个重要的衡量标准。
It’s common to compare the automatic, intuitive part of the decision-making brain with the part that’s rational and deliberate.
人们通常将决策大脑中自动、直觉的部分与理性、深思熟虑的部分进行比较。
But a psychologist testing for this among professional survey takers may very well be on a fool’s errand.
但心理学家在专业调查者中进行这一测试很可能是徒劳的。
Katie Hays, 28, makes about $200 a week Turking in Biloxi, Mississippi, and has observed a change in her own survey performance.
28 岁的凯蒂·海斯 (Katie Hays) 住在密西西比州比洛克西的 Turking，每周收入约 200 美元，她观察到自己的调查表现发生了变化。
“It’s hard to reproduce a gut response when you’ve answered a survey that’s basically the same 200 times,” she said.
“当你回答了 200 次基本上相同的调查时，很难重现直觉反应，”她说。
“You kind of just lose that freshness.”
“你就失去了新鲜感。”
Recently, Rand recruited a group of college students and Turkers — 5,831 total subjects — to perform a series of experiments testing whether cooperative behavior that’s successful in daily life will spill over into what’s known as a Public Goods game.
最近，兰德招募了一群大学生和土耳其人——总共 5,831 名受试者——进行一系列实验，测试日常生活中成功的合作行为是否会蔓延到所谓的公共物品游戏中。
In the game, participants were given a small amount of money and asked to make a choice: how much cash do they keep and how much gets contributed to the common pool, which benefits other players.
在游戏中，参与者获得少量资金，并被要求做出选择：保留多少现金以及向公共池贡献多少资金，这对其他玩家有利。
He then made the players more or less inclined to rely on their gut responses by forcing them to answer quickly or to stop and consider their choices.
然后，他通过强迫玩家快速回答或停下来考虑他们的选择，让他们或多或少倾向于依赖自己的直觉反应。
Among college students, the pattern was clear.
在大学生中，这种模式很明显。
When forced to answer quickly, they were more likely to contribute to the common good; the more time they had to deliberate, the more they hoarded for themselves.
当被迫快速回答时，他们更有可能为共同利益做出贡献；他们需要深思熟虑的时间越多，他们为自己囤积的时间就越多。
But experienced Turkers behaved differently.
但经验丰富的土耳其人的行为却有所不同。
Among them, the impulse to share had largely disappeared, likely because they knew the game, and had learned that sharing was not a good strategy.
其中，分享的冲动基本上消失了，可能是因为他们了解游戏，并且知道分享不是一个好的策略。
The study was published in the journal Nature Communications in April 2014.
该研究发表在 2014 年 4 月的《自然通讯》杂志上。
There are two critical points here as they relate to Mechanical Turk.
这里有两个与 Mechanical Turk 相关的关键点。
The first is that frequent Mechanical Turk workers are fluent in these experiments on arrival.
首先，机械土耳其人的频繁工作人员在抵达时就能够熟练地进行这些实验。
They know how to play the game.
他们知道如何玩游戏。
But also, perhaps more importantly, their natural human impulses from daily life, as they apply to the game, no longer exist.
但也许更重要的是，他们日常生活中自然的人类冲动（正如它们应用于游戏一样）已不复存在。
“If you’re running social psychology studies on Turk, watch out, because [the subjects] have gotten experienced, and that can change effects,” Rand said.
“如果你正在对土耳其人进行社会心理学研究，请小心，因为[受试者]已经变得经验丰富，这可能会改变效果，”兰德说。
“So if you run my experiment on Turk right now, you won’t get any effect.
“所以如果你现在在特克身上进行我的实验，你不会得到任何效果。
Which sucks for me.”
这对我来说很糟糕。”
To be clear, extreme experience isn’t always a problem, Rand said.
兰德说，需要明确的是，极端的经历并不总是一个问题。
There are some psychological tests that are so robust that no amount of experience will override the effect, said Jesse Chandler, a researcher at the University of Michigan’s Institute for Social Research.
密歇根大学社会研究所研究员杰西·钱德勒说，有些心理测试非常有效，任何经验都无法抵消其影响。
The Stroop effect, for example, which involves identifying colors when the color of a word doesn’t match the color spelled out by the text.
例如，斯特鲁普效应涉及当单词的颜色与文本拼写的颜色不匹配时识别颜色。
When the word “red,” for example, is colored green, it takes longer to override the automatic reading of the text and choose green.
例如，当“红色”一词被涂成绿色时，需要更长的时间才能覆盖文本的自动阅读并选择绿色。
“It’s a really powerful effect,” Chandler said.
“这是一个非常强大的效果，”钱德勒说。
“No amount of practice, no amount of awareness will completely override that happening.”
“再多的练习、再多的意识也无法完全推翻这种情况。”
And some puzzles with tricky instructions can benefit from a professional survey taker — someone who’s fluent in the game, said Rand, who notes that he still uses Mechanical Turk often and considers it a great tool, provided that you understand its limitations.
兰德说，一些带有棘手说明的谜题可以从专业的调查员那里得到帮助，他是一个精通游戏的人。他指出，他仍然经常使用 Mechanical Turk，并认为它是一个很好的工具，只要你了解它的局限性。
Can you answer these questions commonly seen on Mechanical Turk?
您能回答 Mechanical Turk 上常见的这些问题吗？
“But if a puzzle has a trick to figure out, once people have seen it a few times, they get it,” Rand said.
“但是，如果一个谜题有一个需要解开的窍门，那么一旦人们看过几次，他们就会明白，”兰德说。
“If they’ve done the task a lot of times before, their intuitions are corrupted, basically.
“如果他们以前已经多次完成这项任务，那么他们的直觉基本上就会被破坏。
That original impulse just isn’t there anymore.”
原来的冲动已经不复存在了。”
It could be argued that the qualities that make these subjects natural and fallible, the very things that make a human human, get swallowed up by experience.
可以说，使这些主体自然而容易犯错的品质，正是使人类成为人类的品质，被经验所吞噬。
Mechanical Turk takes its name from a late-18th century automaton chess player that wowed crowds, outthinking humans and defeating nearly every opponent it faced, including Napoleon Bonaparte and Benjamin Franklin.
Mechanical Turk 的名字取自 18 世纪末的一位自动国际象棋棋手，他令众人惊叹不已，其思维超越人类，击败了它所面对的几乎所有对手，包括拿破仑·波拿巴和本杰明·富兰克林。
But it had a secret.
但它有一个秘密。
After some 50 years of winning, it was revealed that a hidden chess master was concealed inside, working the pieces.
经过大约 50 年的胜利，人们发现里面隐藏着一位隐藏的国际象棋大师，正在处理棋子。
There was a human behind the machine.
机器后面有一个人。
Amazon’s modern-day Mechanical Turk has humans too, gobs of them, powering its virtual machine.
亚马逊现代的土耳其机器人也有人类，为它的虚拟机提供动力。
But is it possible it has its own dirty secret?
但它有可能有自己肮脏的秘密吗？
If so, it’s this: The humans behind the machine have essentially become machines.
如果是这样的话，那就是：机器背后的人类本质上已经变成了机器。
It was in 1675 that physicist Sir Isaac Newton, in a letter, famously scrawled the words, “If I’ve seen further, it is by standing on the shoulders of giants.” Scientists are forever teetering on the shoulders of other scientists, and nowhere is that better represented than by the well-tested paradigms used by researchers who deal in surveys.
1675 年，物理学家艾萨克·牛顿爵士在一封信中潦草地写下了一句著名的话：“如果说我看得更远，那是因为我站在巨人的肩膀上。”科学家永远在其他科学家的肩膀上摇摇欲坠，没有什么比从事调查的研究人员使用的经过充分测试的范式更好地代表这一点了。
Games, puzzles or individual questions that have proven effective at measuring something are reused constantly with minor modifications.
已被证明可以有效测量某些事物的游戏、谜题或个人问题可以不断地重复使用，并进行细微的修改。
When something has been proven to work, there’s value in not reinventing the wheel.
当某件事被证明有效时，不重新发明轮子就有价值了。
Back in Booneville, Mississippi, Marshall pulls up a screen with one of the most commonly used study questions: “A bat and a ball cost $1.10.
回到密西西比州的布恩维尔，马歇尔打开一个屏幕，上面有一个最常用的研究问题：“一根球棒和一个球的价格为 1.10 美元。
The bat costs one dollar more than the ball.
球棒比球贵一美元。
How much does the ball cost?”
这个球要多少钱？”
Think about it for a second.
想一想。
The answer is 10 cents, right?
答案是10美分，对吧？
Nope.
没有。
It’s a nickel.
这是一个五分钱。
“The first time I saw this question, it would have tripped me up,” Marshall said.
“当我第一次看到这个问题时，我一定会被它绊倒，”马歇尔说。
“It’s one of those things — it’s supposed to challenge your critical thinking.
“这是其中之一——它应该挑战你的批判性思维。
It doesn’t do that for me anymore.”
它不再为我做这样的事了。”
The train tracks cut through downtown Booneville, where Sarah Marshall lives and works.
火车轨道穿过布恩维尔市中心，萨拉·马歇尔在那里生活和工作。
Photo by Mike Fritz
迈克·弗里茨摄
Nobel Laureate Daniel Kahneman, a behavioral economist at Princeton University, wrote about this puzzle in his book, “Thinking Fast and Slow.” “Many thousands of university students have answered the bat-and-ball puzzle and the results are shocking,” he wrote.
诺贝尔奖获得者、普林斯顿大学行为经济学家丹尼尔·卡尼曼在他的著作《快与慢思考》中讨论了这个难题。 “成千上万的大学生已经回答了球棒和球的谜题，结果令人震惊，”他写道。
“More than 50 percent of students at Harvard, MIT, and Princeton gave the intuitive — incorrect — answer.” At less selective universities, he continued, more than 80 percent got it wrong.
“哈佛大学、麻省理工学院和普林斯顿大学超过 50% 的学生给出了直观的——错误的——答案。”他继续说道，在选择性较低的大学中，超过 80% 的人都答错了。
“The distinctive mark of this easy puzzle,” he wrote, “is that it evokes an answer that is intuitive, appealing, and wrong.”
“这个简单谜题的独特之处在于，”他写道，“它能唤起一个直观、吸引人但错误的答案。”
Marshall answers it correctly every time.
马歇尔每次都正确回答。
That’s because it appears, on average, in one survey she sees every single day.
这是因为，平均而言，它出现在她每天看到的一项调查中。
She not only knows the answer, but also the two questions that will almost inevitably follow.
她不仅知道答案，还知道接下来几乎不可避免的两个问题。
The first involves using machines to make widgets, and the second is about the rate that lily pads expand in a pond.
第一个涉及使用机器制造小部件，第二个涉及睡莲在池塘中膨胀的速度。
“They’re always in the exact same order and the exact same questions,” she says.
“他们总是按照完全相同的顺序和完全相同的问题，”她说。
Marshall admits that in her line of work, she sometimes observes her behavior becoming mechanical.
马歇尔承认，在她的工作中，她有时会发现自己的行为变得机械化。
“You just get in a motion,” she said.
“你只要行动起来，”她说。
“And you’ll see particular questions and it’s like, if I see the same block of questions twice on the same day, I even know the pattern for my answers.”
“你会看到特定的问题，就像，如果我在同一天两次看到同一组问题，我什至知道我的答案的模式。”
To get to Marshall’s house in Booneville, you drive south from Corinth on 45, past the Thrasher Baptist Church, the Ol’ 45 One Stop and acres of empty fields.
要到达马歇尔在布恩维尔的家，您可以从科林斯沿 45 号公路向南行驶，经过 Thrasher 浸信会教堂、Ol’ 45 One Stop 和数英亩的空地。
The town is partly rundown, with trailers overgrown with weeds, a gutted town theater and abandoned stores with scratched out signs.
小镇部分已经破败不堪，拖车上杂草丛生，小镇剧院被毁，废弃商店的招牌被划破。
But the 24-year-old has managed to eke out a living independent from Booneville.
但这位 24 岁的年轻人已经成功地摆脱了布恩维尔的束缚，维持着独立的生活。
Inside the apartment, duct tape holds together an old air conditioning unit; a red blanket pinned to the window is a makeshift curtain.
公寓内，用胶带将一台旧空调固定在一起；钉在窗户上的红色毯子是临时的窗帘。
Marshall’s husband, Isaiah, paints cars at a nearby auto-body shop, and she “Turks” to help pay the bills.
马歇尔的丈夫以赛亚 (Isaiah) 在附近的一家汽车修理店喷漆，而她则是“土耳其人”来帮助支付账单。
What began as a hobby turned into a job after Grayson was born.
格雷森出生后，最初的爱好变成了工作。
That’s when she realized she could make more money Turking than she was making as a line cook in a nearby deli — and not come home with aching legs, smelling like sandwiches.
就在那时，她意识到，她在 Turking 赚的钱比她在附近一家熟食店做一线厨师赚的钱还要多，而且回家时不用双腿酸痛，闻起来像三明治。
Plus, she could spend her days at home with her son.
另外，她可以在家和儿子一起度过日子。
It’s noon, and she’s now working on a UC San Diego survey — a game that pits her against another worker, who lobs nasty comments at her as they play.
现在是中午，她正在参与加州大学圣地亚哥分校的一项调查——这是一个让她与另一名员工对决的游戏，后者在玩游戏时对她发表了恶毒的评论。
“I think they might be testing the psychological impact of trash talk,” Marshall says.
“我认为他们可能正在测试垃圾话的心理影响，”马歇尔说。
She suspects the other worker is actually a “bot” pretending to be a person.
她怀疑另一名工人实际上是一个冒充人的“机器人”。
To test her theory, she throws out a nonsensical comment.
为了检验她的理论，她抛出了一个无意义的评论。
“I enjoy walruses,” Marshall types.
“我喜欢海象，”马歇尔说道。
The response from the other player: “Also, I’m taking that bonus money.
另一位玩家的回应是：“另外，我也拿了奖金。
And you’re going to lose.”
而你就会输。”
Under the desk, Grayson sneezes.
格雷森在桌子底下打了个喷嚏。
It’s a slow day.
这是缓慢的一天。
Marshall has completed five surveys so far, and made $8.35.
到目前为止，Marshall 已完成五项调查，收入为 8.35 美元。
In a message, she alerts the researcher, “I knew I was playing a robot.”
她在一条消息中提醒研究人员，“我知道我在扮演一个机器人。”
“I can’t comfortably separate my honesty from money, because at the end of the day, all you really have is your integrity,” she says.
“我无法轻松地将诚实与金钱分开，因为归根结底，你真正拥有的就是你的诚实，”她说。
“And if you are willing to sell that for a dollar, it says a lot about the kind of person you are.”
“如果你愿意以一美元的价格出售它，这很大程度上说明了你是一个什么样的人。”
Sarah Marshall dresses her son, Grayson, between Mechanical Turk HITs.
莎拉·马歇尔 (Sarah Marshall) 给她的儿子格雷森 (Grayson) 穿上了 Mechanical Turk HIT 的服装。
Marshall prides herself on being honest.
马歇尔为自己的诚实感到自豪。
There are jobs she won’t do.
有些工作她不会做。
She won’t engage in what she calls the “seedy parts of the Internet,” like the HITS that pay workers to write a five-star review for a restaurant they’ve never dined at or an app they’ve never used.
她不会参与她所说的“互联网的肮脏部分”，比如付费给员工为他们从未吃过饭的餐厅或他们从未使用过的应用程序撰写五星级评论的 HITS。
She has other limits too: For example, no more jobs involving cartoon characters having sex: “I’ve seen Inspector Gadget doing very dirty things to Penny,” she says.
她也有其他限制：例如，不再有涉及卡通人物发生性行为的工作：“我见过加吉特督察对佩妮做了非常肮脏的事情，”她说。
As for surveys, when she thinks her experience might be compromising the data, she sends the research team an alert: “I realized there was some deception,” she’ll write.
至于调查，当她认为自己的经历可能会损害数据时，她会向研究团队发送警报：“我意识到存在一些欺骗，”她写道。
Or: “I’ve seen these questions before.”
或者：“我以前见过这些问题。”
“When I start to feel like I’m becoming really mechanical or like I am a machine who takes surveys, I’ll say, ‘I’m going to stop now and regain my humanity,’” Marshall says.
马歇尔说：“当我开始感觉自己变得非常机械，或者就像我是一台进行调查的机器时，我会说，‘我现在要停下来，重新获得人性。’”
“And I’ll start again tomorrow when I start to feel like a person again.”
“明天当我再次感觉自己像个正常人时，我就会重新开始。”
Sometimes Rand wishes he and other early adopters had never let the Mechanical Turk genie out of the bottle.
有时，兰德希望他和其他早期采用者从未将机械土耳其人的精灵从瓶子里放出来。
“Everything that I study is public goods, right?” he said.
“我研究的一切都是公共产品，对吗？”他说。
“Exploitable resources and how you get people to not overexploit resources.
“可利用的资源以及如何让人们不过度利用资源。
But in some sense, Mechanical Turk is just that.
但从某种意义上说，Mechanical Turk 就是这样。
Because it got so popular, it’s overexploited, and now it doesn’t work for the things that I was originally wanting to use it for.
因为它太流行了，所以它被过度开发了，现在它不能满足我最初想要用它做的事情了。
There’s a little bit of ‘Man, couldn’t we have just kept our mouth shut and kept it as this nice, clean thing for ourselves?’”
有一点‘伙计，我们就不能闭嘴并把它保留为我们自己的美好、干净的东西吗？’”
But, as Rand acknowledges, if it wasn’t him, some other researcher would have discovered Mechanical Turk eventually and let it loose on the masses.
但是，正如兰德所承认的那样，如果不是他，其他一些研究人员最终也会发现 Mechanical Turk 并将其传播给大众。
There was no keeping it contained.
没有办法控制住它。
Not long ago, Chandler and other researchers tackled head on the question of extreme experience, which they refer to as “nonnaïveté.” Early in their article, published March 2014 in the journal Behavioral Research Methods , they sum up their concerns about experience: Turkers repeat the same or similar studies, they share information through discussion boards, and plug-ins allow them to complete the tasks of favored requesters.
不久前，钱德勒和其他研究人员直面了极端体验的问题，他们称之为“nonnaïveté”。在 2014 年 3 月发表在《行为研究方法》杂志上的文章的早期，他们总结了他们对体验的担忧：土耳其人重复相同或相似的研究，他们通过讨论板共享信息，插件允许他们完成受青睐的请求者的任务。
“While participant nonnaïveté can also be an issue in traditional participant pools, M.
“虽然参与者不幼稚也可能是传统参与者池中的一个问题，但 M.
Turk workers might share information in a more systematic, permanent, and searchable manner, with more dramatic consequences for data validity,” the article reads.
土耳其工人可能会以更系统、永久和可搜索的方式共享信息，从而对数据有效性产生更显着的影响，”文章写道。
The team took a sample of 16,408 Mechanical Turk HITs, completed by 7,498 workers and found that a fraction of the workers had completed a disproportionate number of the HITS: 10 percent of workers had completed 41 percent of the HITs.
该团队对由 7,498 名工人完成的 16,408 个 Mechanical Turk HIT 进行了抽样调查，发现一小部分工人完成了不成比例的 HIT 数量：10% 的工人完成了 41% 的 HIT。
(Study authors refer to this group as “super Turkers.”) However, in another sample of 132 published papers, the team found that only 5 percent of researchers had addressed worker experience or “nonnaivete” as a possible limitation of the study.
（研究作者将这一群体称为“超级土耳其人”。）然而，在 132 篇已发表论文的另一个样本中，研究小组发现，只有 5% 的研究人员将工人经验或“nonnaivete”视为研究的可能限制。
“We found that researchers basically ignore this,” Chandler said.
“我们发现研究人员基本上忽略了这一点，”钱德勒说。
“They’ll run multiple studies, and there will be no mention of whether they excluded duplicate workers.”
“他们将进行多项研究，并且不会提及他们是否排除了重复的工人。”
Why aren’t researchers more forthcoming about this information?
为什么研究人员不更愿意透露这些信息？
The sheer number of experienced respondents is an issue that is currently not appreciated by researchers, the study concludes, along with a recommendation: that researchers avoid commonly-used paradigms and, “at minimum, make an effort to measure whether participants have participated in similar experiments before.”
该研究得出的结论是，目前研究人员并没有意识到经验丰富的受访者数量庞大，并提出建议：研究人员应避免使用常用的范式，“至少要努力衡量参与者之前是否参与过类似的实验。”
Researchers need to think carefully about how they’re using Mechanical Turk and whether it’s appropriate for their studies at all, said Gabriele Paolacci, one of the study authors, and an assistant professor of marketing at Erasmus University Rotterdam in the Netherlands.
研究作者之一、荷兰鹿特丹伊拉斯姆斯大学营销学助理教授 Gabriele Paolacci 表示，研究人员需要仔细考虑他们如何使用 Mechanical Turk，以及它是否适合他们的研究。
If researchers receive poor data after using the forum mindlessly, relying on common questions or underpaying workers, “they shouldn’t blame the market, they should blame themselves,” he said.
如果研究人员盲目地使用论坛、依赖常见问题或支付过低工资后收到的数据很差，“他们不应该责怪市场，而应该责怪自己，”他说。
Early results by the team suggests another potentially interesting finding.
该团队的早期结果表明了另一个可能有趣的发现。
Turkers seem more likely to provide false negatives – failing to observe a phenomenon that exists — than false positives — falsely observing something that doesn’t exist.
Turkers 似乎更有可能提供假阴性（未能观察到存在的现象），而不是假阳性（错误地观察到不存在的事物）。
(An example of a false positive would be a study that shows a relationship between vaccines and autism that doesn’t really exist.
（假阳性的一个例子是一项研究表明疫苗和自闭症之间并不存在真正的关系。
A test that fails to show the effectiveness of a successful drug would be a false negative.)
未能显示成功药物有效性的测试将是假阴性。）
“It’s debatable which one’s worse,” Chandler said.
“哪一个更糟糕是有争议的，”钱德勒说。
“But we could imagine an alternate world where workers follow requesters, and they want to be helpful, so they learn their hypotheses and create fictional data to support that.
“但我们可以想象一个替代世界，工作人员会跟随请求者，他们希望提供帮助，因此他们了解自己的假设并创建虚构的数据来支持这一点。
That’s the more troubling outcome.”
这是更令人不安的结果。”
“Historically, science has treated false positives as more worrisome,” Paolacci said.
“从历史上看，科学一直认为误报更令人担忧，”保拉奇说。
Numerous studies in various academic fields — social psychology, cognitive psychology and clinical psychology, among others — have shown that Turkers provide data of equal or better quality than more traditional participant pools.
社会心理学、认知心理学和临床心理学等各个学术领域的大量研究表明，Turkers 提供的数据质量与更传统的参与者池相同或更好。
They’re also more diverse.
他们也更加多样化。
Susan Fiske, a professor of psychology at Princeton University who has used Mechanical Turk “dozens of times” in her lab, said, compared to undergraduates, “it’s way more representative of American people.”
普林斯顿大学心理学教授苏珊·菲斯克 (Susan Fiske) 在她的实验室中使用过 Mechanical Turk“数十次”，她表示，与本科生相比，“它更能代表美国人民。”
“From my point of view as a social psychologist, this is so much better than running college students,” Fiske said.
“从我作为一名社会心理学家的角度来看，这比管理大学生要好得多，”菲斯克说。
“And hands down so much better than college sophomores.”
“而且毫无疑问比大学二年级的学生要好得多。”
A study slated for publication in the journal, Field Methods, found that Turkers, compared to people who are asked to participate in surveys via Google ads, provided better data, fewer “don’t know” responses and more disclosure.
一项计划发表在《现场方法》杂志上的研究发现，与那些被要求通过谷歌广告参与调查的人相比，土耳其人提供了更好的数据、更少的“不知道”回答和更多的披露。
They were also more likely to provide their cell phone numbers and report their household incomes, said Christopher Antoun, a doctoral candidate in survey methodology at the University of Michigan, and the study’s lead author.
该研究的主要作者、密歇根大学调查方法学博士生克里斯托弗·安图恩 (Christopher Antoun) 表示，他们也更有可能提供手机号码并报告家庭收入。
“We were surprised by it,” Antoun said.
“我们对此感到惊讶，”安图恩说。
“My intuition going into it was that these workers might provide low-quality data.
“我的直觉是这些工作人员可能提供低质量的数据。
They might rush through to get to the next task.
他们可能会匆匆忙忙地完成下一个任务。
But there were fewer ‘don’t know’ answers provided by M Turk recruits.”
但 M Turk 新员工提供的‘不知道’答案较少。”
Independent reporting by the NewsHour supports that.
《新闻一小时》的独立报道支持了这一点。
To report this story, we decided to go “meta” and post our own HITs to Mechanical Turk.
为了报道这个故事，我们决定走向“元”并将我们自己的 HIT 发布到 Mechanical Turk。
First, we sought experienced workers to interview about their own experiences in this line of work.
首先，我们寻找有经验的工人来采访他们自己在这方面工作的经历。
We offered them no money, though Amazon did charge us 5 cents to post.
我们没有给他们钱，尽管亚马逊确实向我们收取了 5 美分的发帖费用。
Since we were particularly interested in those with experience, we asked for workers with a 98 percent approval rating and an Amazon-awarded “Master’s degree.” (The Master’s degree, we would later learn, is like the unicorn of Mechanical Turk.
由于我们对有经验的人特别感兴趣，所以我们要求员工的支持率达到 98%，并且拥有亚马逊授予的“硕士学位”。 （我们后来了解到，硕士学位就像土耳其机械公司的独角兽。
No one knows why or when Amazon awards it.
没有人知道亚马逊为何或何时授予该奖项。
But once acquired, it’s a window into a whole new dimension of HITs, and that means more earnings opportunities.)
但一旦被收购，它就是进入 HIT 全新维度的窗口，这意味着更多的盈利机会。）
In our second HIT, we asked workers with a 95 percent approval rating who had completed more than 5,000 HITs to rank the questions they most commonly see repeated.
在第二次 HIT 中，我们要求支持率达到 95%、已完成 5,000 多项 HIT 的员工对他们最常看到重复的问题进行排名。
No Master’s required this time.
这次不需要硕士。
For this one, we paid each worker $1.
为此，我们向每位工人支付 1 美元。
The responses flooded in — 100 answers in 28 minutes.
回复如潮水般涌来——28 分钟内就有 100 个回复。
Like Marshall, the workers we interviewed said they recognized many repeated questions and were aware of attempts to manipulate them.
与马歇尔一样，我们采访的工人表示，他们认识到许多重复的问题，并意识到有人试图操纵这些问题。
But many also reported a degree of pride in their work, despite the tedium and lousy pay.
但许多人也对自己的工作感到一定程度的自豪，尽管工作单调乏味，工资也很低。
And, notably, despite the lack of oversight, they weren’t even tempted to game the system.
值得注意的是，尽管缺乏监督，他们甚至没有想玩弄这个系统。
“I think most of us who do this with any form of seriousness realize that we’re representing not only all of the Turkers, but M.
“我认为，我们大多数以任何形式认真地这样做的人都意识到，我们不仅代表了所有土耳其人，还代表了 M.
Turk as well,” said Clay Hamilton of Chenango Forks, New York, who Turks to pay for heating fuel and family travel.
纽约希南戈福克斯的克莱·汉密尔顿（Clay Hamilton）说道，土耳其人也负责支付取暖燃料和家庭旅行费用。
He estimates he’s done about 40,000 academic surveys.
他估计他已经做了大约 40,000 项学术调查。
“If we start providing garbage information and garbage data, the work is going to slowly dry up.”
“如果我们开始提供垃圾信息和垃圾数据，这项工作就会慢慢枯竭。”
In fact, there’s a whole underground community made up of watchdogs for this sort of thing.
事实上，有一个由此类事情的看门狗组成的整个地下社区。
William Little is a moderator on Turker Nation, one of the many online subcommunities that have grown out of Mechanical Turk.
William Little 是 Turker Nation 的版主，Turker Nation 是由 Mechanical Turk 发展而来的众多在线子社区之一。
Among Turker Nation’s official rules, he said: “No disclosure or discussion of attention memory checks.
在 Turker Nation 的官方规则中，他说：“不得披露或讨论注意力记忆检查。
No discussion of survey content, period.
没有讨论调查内容，期间。
That can affect the results.”
这可能会影响结果。”
There’s no escaping the fact that doing thousands of experiments could lead to behavior that’s more automated than spontaneous — driven more by rote memory than gut instinct.
无可回避的事实是，进行数千次实验可能会导致行为更加自动化，而不是自发——更多地是由死记硬背而不是本能驱动。
But the ultimate effect on the workers, and on the research, may be more nuanced.
但对工人和研究的最终影响可能更加微妙。
Many Turkers report an increasing degree of self awareness that they attribute directly to their jobs.
许多土耳其人表示，他们的自我意识不断增强，这直接归功于他们的工作。
Marshall says the surveys she has completed have inspired her to become more educated and more grounded in her own beliefs.
马歇尔说，她完成的调查激励她接受更多教育，更加坚定自己的信念。
She has been asked to think deeply about things she might not otherwise consider.
她被要求深入思考她可能不会考虑的事情。
And what could be more fundamentally human?
还有什么才是更本质的人性呢？
Hamilton summed it up this way: “I’d always considered myself a conservative, but the more of these surveys I’ve taken, I’ve realized I’m more of a moderate liberal.
汉密尔顿这样总结：“我一直认为自己是保守派，但随着我接受的调查越多，我意识到自己更像是一个温和的自由派。
I’d always considered myself a Christian, but now I consider myself more of an agnostic.
我一直认为自己是基督徒，但现在我认为自己更像是一个不可知论者。
You just can’t fill out 40,000 surveys and not get a better sense of what you do and don’t think.”
你不可能在填写了 40,000 份调查问卷后，才能更好地了解自己做了什么、没有想什么。”
Scientists undoubtedly will continue to wrestle with questions about Mechanical Turk and its use in academic research.
毫无疑问，科学家们将继续努力解决有关 Mechanical Turk 及其在学术研究中的应用的问题。
Is the machine duping researchers — the way its chess-playing namesake once duped 18th-century audiences?
这台机器是否在欺骗研究人员——就像它的下棋同名机器曾经欺骗 18 世纪的观众一样？
Are Turkers misleading scientists and corrupting their science?
土耳其人是否在误导科学家并破坏他们的科学？
Or, in a universe of imperfect study participants, is Mechanical Turk the best option available?
或者，在一个不完美的研究参与者的世界中，Mechanical Turk 是最好的选择吗？
Then again, maybe Mechanical Turk isn’t a corrupt model at all.
话又说回来，也许 Mechanical Turk 根本就不是一个腐败的模型。
Isn’t it possible that, with the help of its savvy group of workers, the research it feeds is actually as strong, or stronger, than ever?
难道在精明的工作人员的帮助下，它所提供的研究实际上比以往任何时候都更强大，甚至更强吗？
How ironic it would be to find out the “robots” in the machine have become better than anyone at telling us about the human condition.
如果发现机器中的“机器人”比任何人都更擅长告诉我们人类的状况，那将是多么讽刺的事情。
© 1996 - 2016 NewsHour Productions LLC.
© 1996 - 2016 NewsHour Productions LLC。
All Rights Reserved.
版权所有。
Support the kind of journalism done by the NewsHour...
支持 NewsHour 所做的新闻工作......
Become a member of your local PBS station .
成为当地 PBS 电视台的会员。