Human Intelligence Tasks


Whilst the concept of crowdsourcing is not particularly new, human intelligence tasks were first popularised by Amazon with their Mechanical Turk web service. The online crowdsourcing service was created out of Amazon’s need for an efficient way to sort duplicate or similar product entries in their ecommerce database. Amazon created a platform that allowed anyone to register as a worker on their website to get paid (often only a few cents) for the completion of HITs (Human Intelligence Tasks). Although initially developed for Amazon’s own purposes, thousands of companies and individuals now submit tasks to Mechanical Turk each day.

Similar services

Mechanical Turk has received a lot of renewed publicity (both positive and negative), perhaps due in part to two new services that leverage the platform. Crowdflower is a web application that helps users of Mechanical Turk and similar platforms to “significantly reduce error rate, improve performance and get results quickly.”  Gambit has also partnered with Crowdflower to let online gamers or community members earn virtual currency by doing simple human intelligence tasks.

Pioneering work by the legendary Luis von Ahn on leveraging people to generate useful metadata through games has also captured the imaginations of many with the possibilities human computation.

Numerous other services have been developed over the last few years that compete with Mechanical Turk such as LiveworkOdeskCloudcrowdShort Task and Humangrid.

Common tasks

Some of the common tasks found on Mechanical Turk and similar services include:

  • Transcribing recorded audio into written text.
  • Writing a product review or commenting on a blog article.
  • Rating the quality of an article or product.
  • Rating search results to help influence search engine relevance.
  • Filtering articles or images that include inappropriate content.

There are also many more task types that are specific to the requesting organisation’s business requirements. Some examples of novel tasks that require human intelligence are “naming colours displayed on the screen” and “computer vision training“.

Mobile human intelligence services

A few organisations have recently launched initiatives to support the completion of simple tasks on mobile devices as opposed to on internet enabled computers.

iPhone Applications

Crowdflower’s Givework and The Extraordinaries are both examples of iPhone applications that facilitate volunteers completing simple tasks for a good cause. The types of tasks they cater for are similar to those above.

Simple SMS / text message tasks

Txteagle is a service aimed specifically at workers in developing nations. The service enables simple tasks to be completed via text messaging. Some of the tasks currently being completed include:

  • Listening to a voice message and transcribing it into text.
  • Phoning a specified number to train a computer to understand local dialects.
  • Translating short phrases for software localisation.

Some of the common human intelligence tasks found on services like Mechanical Turk are not suitable for completion on mobile phones due to the small screen, limited interface and low bandwidth. Although the simplicity of the Txteagle service caters to a massive market of entry level mobile phone users, it is especially limiting compared to the iPhone examples in that forms, rich media, validation and embedded logic are not supported.

How Mobenzi Intelligence is different

Mobenzi Intelligence leverages the mobile form technologies developed for Mobenzi Researcher and therefore supports fairly complex tasksintermittent network connectivity and embedded logic. Although not as widely supported as SMS, over 300 different models of mobile phones currently support the application – with more planned. Mobenzi Intelligence agents – people who sign up to download the Mobenzi Intelligence application – receive and complete tasks via their mobile phones.

The software communicates via the mobile internet which allows instructive text and other content to be downloaded at a fraction of the cost of an sms.

The following factors affect the selection of appropriate tasks for Mobenzi Intelligence agents.

  • Tasks should take only a few minutes to complete.
  • Some phones might have very small screens making large bodies of text difficult to read.
  • Embedded images and audio are not yet supported but will be in coming months.
  • Mobenzi Intelligence agents will not undergo detailed training for specific tasks.
  • Agents will perform tasks wherever they choose – at home or on public transport.
  • Completing a task should be very simple for someone with limited education.
  • A task may be made up of a series of instructions and questions.
  • variety of question types are supported, including single option lists, multi option lists, dates, text etc.

Types of tasks Mobenzi Intelligence agents can complete

Taking the above points into account, we have outlined some initial ideas for the types of tasks that would be suited to Mobenzi agents.

Text to form (data extraction)

These types of tasks would involve sending a string of free text (e.g. taken from an SMS, Email or Tweet received by a business as part of its normal operations) to an agent together with a predefined classification form made up of instructions and questions. The agent will read the text and complete the form based on the content of the text. Two simple ideas for making use of this service include the structuring and sorting of free text SMS survey responses offered by our Mobenzi Researcher service and rating the sentiment of Tweets.

Audio to form (data extraction)

These types of tasks would involve sending a single audio clip to an agent together with a classification form. The agent will listen to the audio and complete the form based on the content of the audio. This service would allow organisations to record simple spoken audio from their customers or research participants and then send the recording to Mobenzi Intelligence for structuring and sorting. Some application ideas include supporting the conduction of audio (IVR) research studies and rating the sentiment or classifying a recorded call center conversation.

Textual translation

These tasks would involve agents receiving and translating a section of text from one language to another, particularly local languages. (e.g. from Zulu to English). We’ve done some very early piloting of this task and our experiences are outlined in a blog post on crowdsourced mobile translation tasks.

Audio translation

Audio translations would require an agent to listen to an audio clip in one language (e.g. Zulu) and speak it in another (e.g. English). This is the most natural translation process and does not require high levels of written literacy or proficiency in written grammar (it is common that people can speak a language but cannot write well in it). This service could be useful to people learning a new language or trying to communicate with someone in a language they are not themselves familiar with.

Audio transcription

These tasks would involve the transcription of audio clips into plain text. Common applications are the transcription of recorded medical notes, voice messages or call center conversations.