Tin Isles

I’ve read a bit about Amazon’s Mechanical Turk, and it sounds like a very interesting technology. It is a market place to put up little tasks for workers around the world to complete, i.e tag a photo, categorize this dress, find an email for the business, etc. Now I’m wondering what interface you have to create tasks, does it work like a facebook app where I host the logic? Finally got around to reading up on it, so here’s the quick intro for the techie that doesn’t feel like reading the manual. 🙂

Creating some tasks is very simple, you just create a HTML template with placeholders like this: ${placeholder}, and some HTML elements. Upload a CSV to ‘mail merge’ with the template, and workers get displayed 1 task for each row in the CSV. Any data entered into the form fields are saved and available for download.

I came up with an idea for mturk tasks while I was testing an ipad magazine app at work. The app contains bitmaps exported for the PDF files to create the magazines, so there is no metadata to create hotspots on the index page. The idea: put the index pages up on mturk for workers to draw the hotspots. The workers only interact with the HTML in the template, so it is an option to write some javascript to draw the rectangle and store the co-ordinates in hidden form fields.

I want to take 1 index page, and get a rectangle for each page reference. I’m not really using mturk to its fullest if I give 1 worker a JPG, and have the worker draw multiple rectangles. It would be better to “scale” the task over multiple workers. So instead I type all the page numbers into the CSV (this could also be another task!), and each “task” displays the worker a single page number, and asks them to draw the rectangle. Slight problem, what if a page number is listed more than once? Now I have three columns in my csv:

• pageNumber – the page number I want the workers to find
• pageCount – the number of times the page appears
• pageUrl – the location of the scanned page

If a page appears multiple times I give the worker multiple rectangles to highlight the page.

So what’s it look like? The ‘Design’ tab is used to set up the template

I upload the CSV, and the task appears for workers to complete.

I get a UI of the results in real time, the tasks complete pretty quickly. I can click the results button and see a grid of the results. You can also reject the results you see in the grid, and put the task back to the workers for completion.

See the rectangles drawn on the index page:

There is a bit of an overlap on the rectangles, so I wrote a bit of code to ‘split the difference’ of any overlap. See the results here.

Further reading:

1. Creating a HIT Template – a good reference from Amazon on the workings of a HIT created in the UI.
2. API reference – it is possible to host the IFRAME yourself using an ExternalQuestion