The possibility of having Mobenzi agents convert English content into local languages (and vice versa) was one which really excited us.

As with all Mobenzi tasks, we split up a large, complex challenge into small, discrete tasks which can be performed in a few minutes.

Obviously, there are limitless reasons people require translation services. One of our own immediate applications for translation is for surveys conducted using our Mobile Researcher platform. We frequently have clients embarking on multilingual studies who need questions to be available in several local languages (in South Africa alone we have 11 official languages); and need qualitative responses given in other languages available in English for analysis. Normally, surveys are designed, reviewed, debated and eventually finalised in English before they are sent for translation and back-translation. Our objective is to crowdsource this activity instead – making near real-time translations available to survey designers in Mobile Researcher at the click of a button.

In our first trial we took a client’s survey questions which we already had in English and professionally translated Zulu and converted them into two sets of translation tasks: from English to Zulu and vice versa.

As most people don’t speak Zulu (and mine isn’t exactly fluent either), I’ll use a Zulu-to-English example taken from the actual exercise:

Original English (not visible to Mobenzi agent): What is the household’s average monthly income?
Professionally Translated Zulu (shown to Mobenzi agent): Ungayilinganisela kumalini isiyonke imali engenayo ngenyanga?
Mobenzi Agent Translation (from Zulu to English): How much is the monthly income for the household?

Now, clearly the ideal would be for the original phrase to match the version coming back from the Mobenzi agent (and this did happen frequently) but in most cases, it does prove that, for the most part, crowdsourced translation works to the extent that the concept is sufficiently conveyed.

Some of the key challenges and limitations of crowdsourced translation tasks which our agents helped us identify were:

• Lack of context. Just as computer translation algorithms struggle without context – so do people. We noticed that, on several occasions, what agents had responded with made sense if read in isolation, but not within the context of the broader scope. It has to be said that this problem is exacerbated by our approach of segmenting text for translation.
• Language formality. The mobile communication culture is almost exclusively an informal one. Some agents seemed to struggle to snap out of this mode and became frustrated having to use formal language.
• Input mode. Although an obvious observation, translation requires a lot more typing than other types of tasks. Somewhat foolishly, we didn’t enable predictive text on during the first day of our translation trial which didn’t help matters.
• Multiple alternatives. Just as there are a number of ways to phrase a question in English, translation is not a one-to-one mapping. This is not a problem inherent to mobile crowdsourced translation however – two professional translators may also come up with different ways of phrasing something based on experience and personal preference.

An interesting observation is that the value in using crowdsourced translation is not only that one might not necessarily need to use traditional translation services but more that crowdsourced translation can give insight into how “normal” people understand a question in their mother tongue.

Another example:

Original English (not visible to Mobenzi agent): What is the household’s average monthly income?
Professionally Translated Zulu (shown to Mobenzi agent): Ungayilinganisela kumalini isiyonke imali engenayo ngenyanga?
Mobenzi Agent Translation (from Zulu to English): Estimate how much money do you earn a month?

In this case, the agent has incorrectly translated the question back into English by missing that the question wants to know the cumulative income of the household – not just that of the respondent. Now, this might be a mistake by the agent – but a mistake is not, in itself valueless. In this case, it can help a survey designer identify where confusion or areas of potential misunderstanding might occur in the real study. The response from the agent can guide the designer, in collaboration with their translator (professional or otherwise) in refining the question to ensure optimal wording which will make sense to the actual respondents.

We were rather pleased by the quality of translation provided by agents without any kind of moderation, statistical or other methods being applied but clearly there is still a lot of room for improvement in this area. We’ll continue working on some ideas before our main pilot takes place next year.