For the last eight years, I have been working to develop an area of computer science called "human computation," which studies how to harness the combined power of humans and computers to solve problems that would be impossible for either to solve alone. This growing academic field now has an annual workshop, a community with researchers from the top computer science programs in the world, and has directly influenced the popular online trend of crowdsourcing, in which crowds of people are enticed to perform work over the Internet. Subsequent to the development of this area, for example, Amazon created Mechanical Turk, a marketplace for human computation tasks (or “human intelligence tasks” as they call them), which is now used and studied by hundreds of researchers worldwide. Since then, other similar services have emerged where workers are paid to perform micro-tasks that are hard for computers.
An example of human computation is reCAPTCHA, in which people help digitize books by typing CAPTCHAs on the Internet. To date, over 750 million unique people—more than 10% of humanity—have helped transcribe at least one word through reCAPTCHA.
All human computation systems must have a way to motivate the users to participate. In the case of reCAPTCHA, the value proposition is as follows: by typing a CAPTCHA, the user gets access to a desired resource like a free email account or tickets to a concert, and in exchange they perform ten seconds of work that is utilized to help transcribe a book. In the case of Mechanical Turk, users are paid a few cents to perform each task.
A discussion that I've had with multiple people over the last few years is whether systems like Mechanical Turk, in which real money is exchanged, should be legislated so that workers are fairly compensated. You see, the average hourly rate of most workers in such sites is usually well below the minimum wage of most third world countries. As a concrete example, the minimum wage in Guatemala is approximately $1/hour, whereas it's not rare to see tasks on Mechanical Turk in which the effective hourly rate is $0.30/hour. (It's amazing that many of the workers on Mechanical Turk come from inside the United States.) Some labor economists would tell you that this is ok: if people are willing to work for such low rates, who is to stop them? However, most countries have some notion of a minimum wage in their laws, including the United States, so in essence as a country we do not believe in an unregulated labor market.
Recently I have heard more than one company saying something like: "We use Mechanical Turk because otherwise we would have to pay people $7/hour to do this task." In other words: "We use Mechanical Turk to get around the minimum wage laws." As wrong as it may sound to some, this is currently ok. In the United States, "independent contractors" are typically not covered by minimum wage laws, so while I'm not a lawyer I believe using Mechanical Turk to get around minimum wage is as legal as hiring independent contractors instead of full-time employees.
But the question remains: Should sites like Mechanical Turk be regulated? Perhaps not today, but if the Internet or crowdsourcing really is the future of work, we should at least be thinking about it.
Here are some issues that make this complicated:
If it's OK to pay someone nothing to do a task -- say writing a Wikipedia article -- how could it be wrong to pay them 30 cents?
ReplyDeleteAlso, Mechanical Turk workers are paid by the item, not by the hour. Suppose someone works quickly and earns above a minimum wage. Is it then wrong to pay someone else proportionately less when they do less work?
Minimum wage laws sound good at first, but they contain hidden paradoxes and unintended consequences.
Thanks for this post Luis! Expands on some of the themes we discussed on our recent chat with you (I'm linking to this post in the show notes). http://blogs.law.harvard.edu/mediaberkman/2010/07/22/radio-berkman-159-spare-cycles
ReplyDeleteOne of the more interesting ethical questions to me (posed by Jonathan Zittrain in his "Minds for Sale" talk) is the application of distributed labor. In cases unlike ReCaptcha, where disclosure isn't clear, who can say whether a Turker's work could be used for devious purposes? In some cases crowdsourced or game-based labor has been used to identify suspected criminals in surveillance footage. Other cases involve paying commenters to leave favorable reviews. What about spammers employing people to manually crack captchas (as you've highlighted)?
Do we regulate the activity itself? Ignore it until it gets out of hand? Find a technical workaround? Rely on disclosure and personal responsibility? Would be excited to explore these questions more!
@john: I think there is a difference when real money is exchanged because somebody could start making a living out of it.
ReplyDeleteSomething else that complicates the issue: http://www.nytimes.com/2010/04/03/business/03intern.html (it's about whether unpaid internships are legal).
If you are really interested in this issue, maybe you should take a look at WTO regulations about trade in Services (GATS). There you will find out that there is regulation about supply of service " from the territory of one Member into the territory of any other Member".
ReplyDeletebest regards,
Manoel
Is it a bad thing that someone could start making a living off Mechanical Turk? "Uh oh. People are starting to earn real money from this. We'd better shut it down."
ReplyDeleteOf course the idea of a minimum wage law is to force employers to pay people better for the work they're doing. Employers may do that, or they may decide the job isn't economical at the minimum wage and no longer offer the job. So the consequence of requiring higher wages may be higher wages for some and unemployment for others. Minimum wage laws may shut down Mechanical Turk rather than raising wages.
Similarly, if you eliminate unpaid internships, some interns will make more money but there will be fewer internships.
Ok, but you're arguing against minimum wage of all forms since your arguments don't just apply to the Internet. However, minimum wage laws do exist -- even in developing countries.
ReplyDeleteAccording to Wikipedia, only Americans can work at AMT.
ReplyDeleteThat's not what happens in practice. For example, there are quite a few people from Indian.
ReplyDeleteI believe Mechanical Turk has spoiled what is quiet geeky and smart about human computation by paying a derisory amount of money to people who participate. I cannot distinguish much of a difference between one who relies on Mechanical Turk as a source of money and one who is employed by infamous spammers to type captcha.
ReplyDeleteHuman computation is a very smart idea, but I hope it doesn't end up in cheap labor for cheap work.
Imagine the whole linux and free open-source softwares were created out of paying cents per hour to programmers. It would definitely be far from anything people can revere.
There are many situation that the resulting quality would be higher if people had done it for free. I definitely would trust/enjoy less of the reviews on Amazon if people had been paid for what they had written. It is totally fair for Amazon to benefit from the reviews people write for free, because in the first place, Amazon has provided this space that people inside the community can guide each other.
As for legislation, the arguments which justify minimum wage laws are still valid for work being done over internet. Surely there are practical issues when enforcing laws on cyberspace but that cannot challenge the argument behind the law.
You might like to check out Cory Doctorow's 'FTW', a story which happens to discuss the rights of Turks, MMORPG gold miners, unions and workers rights around the world - just finished it myself. It's available in shops and you can also download it for free:
ReplyDeletehttp://craphound.com/ftw/
... as it is distributed under a Creative Commons license.
this is to tell everybody that DasNet is the company that don't pay Local Afghani companies payment for more than 3 months, dont work with them.
ReplyDeleteAs someone who has recently become one of Mechanical Turk's heaviest work requestors, I agree it starts to cross into ethical and politcal boundaries when viewed incontext of traditional laws and policies used to protect human rights, freedoms, and quality of life. But, this is a recurring theme with many internet phenomena. A glaring example, is freedom of speech, which is not universally considered as positively as it is in the US - even by the citizen's of respective countries. The bounary between free speech, slander, and illegal propaganda is culturally dependent.
ReplyDeleteIn my opinion, many of these types of laws were created when physical location was an assumed constraint. People cannot not easily move to a new city, so laws must be instilled to protect a minimum quality of life, or normalize acceptable behavior in a physically co-located community. What happens down the street DOES matter to most people. Most laws and policies you are subject to are based entirely upon where you sleep.
But, the internet makes physical location irrelevant and invalidates a core assumption by which many of these laws were created. There is zero cost to going to a different website, thus there is zero cost to "changing jobs" in a world of professional crowd sourcers. Since there is no involuntary nor unreasonable loss to the individual, there is nothing to protect.
Regulating Mechanical Turk seems with minimum wage laws like a short sighted application of antequated laws design to maintain a physicially co-located society. On the internet people have unlimited options. In the physical world, people are limited by where they can get to and so we need laws to protect them.
If you are a believer that efficient free markets are a good premise to a substainable global society, regulating Mechanical Turk would be very bad.
Also, I STRONGLY believe the bad wages on Mechanical Turk is purely due to a surplus in labor. Simple supply and demand. Not enough paying institutions understand how to use it yet. As more companies learn how to crowd source, demand will increase, and so will the wages and instutions compete for workers. When there are a large number of jobs available (lots of competition), I end up paying a per unit wage similar to hiring domestic contractors.
I agree that these laws are antiquated. That's why I think we should be thinking about this problem.
ReplyDeleteA free market may fix this in the future, but it may also not. Your argument is basically: "currently the wages seem ridiculously low, but if we wait and do nothing, it should be fixed."
At the current rates, I think using Mechanical Turk borders on exploitation.
what about selling crowdsourced output and making a profit out of it? is that illegal?
ReplyDeleteI don't think any of this is illegal. And in most cases, i don't think it should be.
ReplyDeleteExploitation? Hrmmm.. that's where we differ. No one forces people to use MTurk. In fact, from what I can tell... many people just use it as a way to kill time - a financially productive alternative to playing solitaire/minesweeper, or do it while they watch TV. Many HITs could be reasonably turned into GWAPs with some clever twists (and almost wanted to ask you if you had an interest in setting up a GWAP service where companies could format HITs according to semi-proven game styles and you provide the centralized back-end/player pool - ala MTurk).
ReplyDeleteVery few people have made over $1000 during their lifetime of usage, and experienced workers are extremely selective about the work they choose to do. There are some HITs experienced workers refuse to do because the wage is too low, or the rejection rate is so high. They do quite a good job protecting themselves against "exploitation." Turker Nation http://www.turkernation.com/ is a website dedicated to workers reviewing job requestors. Many of those comments can make loose a lot of empathy for workers.
I think one could make a stronger argument that World or Warcraft, Farmville, the lottery, Rolex's ..or GWAPs are more exploitative because they utilize psychological tricks to extract money/labor from people who may not realize they are (at some subconscious level) being involuntarily manipulated. People without a healthy amount self-control, do indeed have unhealthy problems with these with potentially damaging effects on society – and thus rank higher on the list of things that warrant regulation with laws to protect individuals (like gambling). Mechanical Turk is much more explicit and upfront about the total cost/benefit of the transaction for both sides. All the information is clearly visible, and better allows people to make informed decisions about what they are getting into.
Infact... after writing this, I now definitely think GWAPs are more dangerous to society than MTurk because they obfuscates (and in some cases, lies about) what is really going on to users and exploits psychological trickery to manipulate people into potentially unhealthy and self-damaging behavior. =o)
First, the "voluntary" argument is not something our society believes in. Sweatshop workers (even children) are usually there voluntarily.
ReplyDeleteSecond, gambling is illegal because there is money involved. "Addictive" games like those in the Wii or in the XBOX have not been regulated for it (although, I agree maybe they should be in some way).
However, the point remains that there is a huge difference when money starts being involved and people start making a living out of this.
It's true that not many people have made even $1,000 on mturk, but the crazy part is that according to this, mturk is the primary source of income for something like 25% of its workers!
This is not fun and games anymore. It's real.
All of that said, I'm mainly playing devil's advocate here :)
ReplyDeleteThanks for this post, Luis.
ReplyDeleteJohnny Lee, zero switching cost assumes that there is something to switch to that pays better. There is nothing about an unregulated labor market that makes such a guarantee. It isn't necessarily the fault of the worker that an economy has a set of employers who have constructed a very very large pool of jobs requiring no specialized skills. Worker's replaceability means zero switching costs for the worker *and* for the employer (switching to another worker).
Anyways, just because it costs nothing to switch doesn't make it right to pay an extremely low rate, particularly for a job that offers little in the way of enjoyment, intellectual property, and ongoing reputational capital.
To the argument that maybe these jobs would just go away, that's a bigger problem of creating *good* jobs in an economy rather than just justifying convenient exploitation as such.
Luis you are doing a great work. Such as you are trying to solve the problems along with the help of the computers and humans alone which they can not do that.
ReplyDeleteThe switching cost is not zero - every new MTurk task has a learning curve that costs the Turker money because they are less effective at the start, they must search for and discover tasks that are reasonable, etc. In addition, switching is high risk, because switching back is not guaranteed - the hits may be consumed by another Turker.
ReplyDeleteI think over all the "free market" argument is simplistic - every market operates in a regulatory framework of some kind. The issue is, what is the "fair" structure of the regulations. Raising the free market flag avoids focusing on the framework issues.
For example, MTurk does not have a rating system of requesters, nor an arbitration board when a requester denies payment. The lack of theses features biases the system heavily towards the requesters and against the "workers".
With respect to this last comment, read more here:
ReplyDeleteSilberman, M. Six and Ross, Joel and Irani, Lilly and Tomlinson, Bill, "Sellers' problems in human computation markets", HCOMP '10: Proceedings of the ACM SIGKDD Workshop on Human Computation, 2010.
http://doi.acm.org/10.1145/1837885.1837891
Many of you are hinting at an area of great concern, the creation of digital sweatshops. Extremely poor people, children included, may 'agree' to work for paltry remuneration, just as they do today because of depressive economic situations. These are not people who pass idle time on the internet playing games with a purpose. These digital serfs, may not be able to change websites looking for better jobs; they may be placed to work in a digital kiosk with no distraction other that accomplish their tasks. They may have no incentive since their pay wilt be a fixed daily rate and they still will have to work faster.
ReplyDeleteI've also heard the concept that one man's sweatshop is anothers dream job. I would hope that humanity as a whole can adopt some principles that would allow monitoring and prevention of abuse. Technology may offer some solutions. Work units could be cryptographically stamped with the who, what, when, where, for how long, etc. Workers producing these units would also need to have a certain amount of leisure units that could be in the form of relaxation, education, or social interaction. All of this would have to 'add up' before payments are exchanged and sufficient digital evidence would exist to prosecute serious violators.
Great post , made me think of txteagle.
ReplyDeleteRegulation will not solve the problem.
ReplyDeleteThe problem is that payers cannot identify productive workers, as opposed to unproductive ones. If the problem can be solved, it has to be thru creative and innovative entrepreneurs, not the use of law (ie. force).
It is disappointing to see someone as creative as you, Luis, falling back on such a blunt and ineffective tool as regulation to solve real world problems.
Minimum wage laws, and other forms of price control, are well known to fail to achieve the desired goals. The main effect of minimum wage laws is to raise unemployment in the marginal groups that have the lowest productivity (such as teenagers) and also creates adverse selection in the groups that meet the new bar (discrimination).
If you're curious about the economics of minimum wage (an important topic and good introduction to economic analysis in general), see:
ReplyDeletehttp://mises.org/daily/3261
or
http://dumky.posterous.com/repeal-the-minimum-wage
This is one of the rare issues where economists are almost unanimous, yet political incentives and cognitive biases seem to keep us off the correct path...
Thanks for sharing, really like your view. Waiting for some more great articles like this from you in the coming days.
ReplyDeleteYou have done a marvelous job! I am really inspired with your work.
ReplyDelete