An interview with Greg Little, one of the creators of TurKit, a programming interface that makes it easy to write programs that automatically hire people on Amazon’s Mechanical Turk platform to perform complex series of tasks.
What are the challenges specific to writing a program to run « over Mechanical Turk »? What design choices for TurKit were induced by the fact that its programs would be executed by humans?
Typical use of Mechanical Turk involves posting many self-similar tasks, like a separate label-this-image task for a thousand different images. Mechanical Turk has pretty good support for these sorts of tasks. However, it is harder to write programs that must wait for tasks to be done, and create new tasks in response. Programs of this sort may run for hours or days, which makes them difficult to prototype and debug. Many small changes require restarting the process, and waiting more time and spending more money to see the effect. TurKit tries to streamline this process with a clever hack which stores a sequence of important programming operations so that they do not need to be repeated when the program is stopped and re-executed. Of course, this hack means that the program consumes more and more memory the longer it runs, but this is usually an acceptable trade-off for prototyping new processes involving human computation.
Do you foresee Mechanical Turk or similar platforms being used in the future as a regular part of the computing infrastructure? Could they be integrated with existing development and deployment techniques, and what, if anything, would they add to them in terms of capabilities and trade-offs?
Yes. I think that Mechanical Turk and similar systems will act as a bridge between current technology, and the grand vision of true artificial intelligence. Humans are already used for some vision AI tasks, like OCR with reCAPTCHA. I think that processes like writing, data analysis, programming, etc. will also benefit from Mechanical Turk before we have AI capable of accomplishing these tasks. In the near term, I think development environments for word processing and programming will have plugins that allow users to outsource tasks to Mechanical Turk or other online labor markets like oDesk with minimal overhead (e.g. “please look over this paragraph and let me know if it makes sense”). There is already a research prototype for Word that does this, called Soylent (since it is a human powered interface).