Many organisations receive thousands of inbound SMS messages

SMS text messaging is the most widely used data application on the planet, with 2.4 billion active users (Wikipedia). A growing number of organisations are using text messages to communicate with mobile phone users. Although most communications involve distributing information to end users, many organisations receive and process thousands of text messages from end users.

Once an SMS is received by an organisation, its content needs to be analysed and understood to enable reporting or features such as sending a relevant reply SMS. Dealing with a large volume of inbound SMS messages requires automated processing (efficient sorting of messages) to save time and money. Processing these SMS messages manually is often not feasible.

It is difficult for computers to understand messages written by people

Using computer programs to automatically sort sms messages involves either extremely advanced natural language processing (NLP) or end users adopting a specified syntax in their messages (e.g. Sms your NAME, followed by your AGE and then the KEYWORD…) so that the messages can be more easily understood by a computer.

Natural language processing is extremely complex

Natural language processing (NLP) is a field of computer science and linguistics concerned with the interactions between computers and human (natural) languages. Natural language understanding systems convert samples of human language into more formal representations such as parse trees or first-order logic structures that are easier for computer programs to manipulate. (Wikipedia )

Although there are many services like Google SMS and WolframAlpha that make use of NLP concepts, I don’t believe that we will see a generic natural language understanding solution that we can use for SMS messages in the near future, especially one that would cater for the many languages in developing countries. Such systems would still require extensive customisation in order for a computer to understand the meaning of an SMS within a particular context.

Structured syntax is not easy to use

People have invested significantly in making operating systems, computer programs, websites and hardware easy and intuitive to use. For many people however, plain text sms messages are the only interface they will have to online services such as banking, classifieds and social networks for the next few years.

The process of interfacing to a software system via sms is often difficult. The common approach of using structured syntax messages (requiring users to adopt a set of words and rules in their messages) is comparable to entering instructions at the DOS command prompt like many of us used to do 20 years ago. Although the command line as an interface is having somewhat of a comeback (“The Web Browser Address Bar is the New Command Line“),  it is usually only expert users who really make use of most commands.

Many people make use of structured syntax commands with applications like Twitter (“RT” or “@username” ect.), Google (“1USD in ZAR“)  and other services like IRC. But with all of these applications, understanding the structured syntax is not required for new users or the majority of interactions. With many sms systems however, users are required to understand the syntax right from their first interaction. I think it is quite unnatural, unintuitive and difficult for many people to use fixed syntax commands, especially via SMS which is predominantly used for communication between people.

Organisations also need to communicate the rules and language of the system to mobile phone users so that they understand what words they can use and how to structure their messages. It is very difficult to do this via SMS due to limitations in the number of characters in an SMS and the cost of each message, especially if a sequence of messages is required.

Where there are only a few commands or keywords that need to be learned, using structured syntax can work very well. But when SMS systems start offering more interactive services, they will become much more difficult to design and to use.

Perhaps the biggest challenge with any computer based processing of SMS messages is that users often expect that a person will read and understand their message and therefore use informal language (Textese), ask questions or otherwise make spelling and grammatical errors.

People can help computers understand SMS messages

Certain processing tasks, such as understanding an SMS as described above, are still performed better and faster by humans than by computers. But manually processing inconsistently large volumes of such tasks is not feasible for internal staff at many organisations.

This problem inspired us to create Mobenzi, a service that allows organisations to outsource these kinds of tasks to a distributed team of workers who could share the load of many organisations’ processing requirements.

Mobenzi is a software service that empowers people to be rewarded for completing simple tasks on their mobile phones.

We are currently piloting the system and are confident that we will be able to take some live projects on within the next few months. Although there are several business applications we are addressing with Mobenzi, sorting of SMS data is our initial focus.

The concept of human intelligence tasks (simple tasks that computers find difficult) was first popularised by Amazon with their Mechanical Turk web service. We also recently came across Textonic that is attempting to leverage Mechanical Turk for tagging SMS messages in a similar way to Mobenzi. We hope that Mobenzi will be very well suited to processing SMS data since the processing itself can be done on a mobile phone by a Mobenzi agent who speaks the same language as those people sending the SMS messages. We plan on making it very easy to set up new teams of Mobenzi agents in new locations so that the agents will understand the local languages and colloquial terms that are included in the data that needs to be processed.

Mobenzi will allow computer programs that receive SMS messages to seamlessly interface to real people who can interpret free text messages and return information such as categories, tags, and other structured data as part of an automated process. Once an SMS is received by an organisation, the message can be submitted to Mobenzi for processing (via an API call for example). A Mobenzi agent would then be sent the original free text SMS with an associated form (see example below) to extract and structure the relevant information contained in the message – obvious to a human observer but inaccessible to computer systems.

This task illustrates how a simple natural language sms can be processed by a Mobenzi agent to extract structured data from the message.

Find out more about how Mobenzi works.

As part of the current pilot phase, we are assessing the cost and quality issues involved in the completion of tasks. We hope to make the processing service very affordable, especially considering that the requesting system can save costs in the long run through learning from the results generated by Mobenzi.

We hope that Mobenzi will help make many more online services available to people whose only interface to the internet is via SMS.

If you have any specific requirements you’d like to explore using Mobenzi for, please contact us.