Recap Machine Translation Summit 2019 and explore the future of content technology and translation automation.
Subscribe to our blog
MT Summit has come to an end and it’s impossible to leave this three-day conference without being excited about what’s yet to come. From the advances in neural machine translation (NMT), automatic error classification and predictive quality, to non-stop research on content intelligence, we had a glimpse into the future of translation automation.
1. Predictive quality estimation: from theory to practice
The bilingual evaluation understudy (BLEU) scoring methodology continues to be referenced as a quick and low- cost option to evaluate MT progress. Developed by MT engineers, it doesn’t require the involvement of translators, but the results imply there is only one correct translation for a given text, when this is obviously not true. The “approved” translation is then set as a reference, in both source and target language, and the evaluation scores are as high as the similarity between the output and reference translation.
So, while BLEU usually does correlate with human judgement, it does not answer the real question: how good is the actual machine translated content?
Discussions on MT quality metrics are still under way, but the goal is to get to a position where every machine translated content comes with a quality estimation that annotates the MT output and gives it a quality score. Throughout every session, all contributors ended up admitting with nervous giggles that quality can only “truly” be assessed by human review (for now).
2. Neural MT: to APE or not to APE
If I had to choose one buzzword for MT Summit, it would have to be automatic post editing (APE) for neural MT – while not new it is gaining traction. As it starts to be tested by major ecommerce companies, it’s evolving into a real business possibility. But there’s still a question in the air: is it counter-productive or time saving?
APE has followed a similar evolution to machine translation. The aim of an APE system is to correct the mistakes in MT output and generate a "human-like'" post-edited machine translation (PEMT). APE can improve the machine translated content or adapt it to a specific domain. For example, in proprietary systems, it can come in handy as a way to learn from human corrections and avoid recurring errors, teaching the system what not to do. Even with a high-quality MT engine, automatic post editing can help to reduce the human effort on post-editing.
APE can also be incorporated into computer-assisted translation (CAT) tools to aid content reviewers in post editing. However, test results of this application have proved it to be counter-productive, as linguists tend to spend less effort on segments that have been rated 90 percent (or more) by the APE tool, and move on to next segment. The problem is that 90 percent score may or may not be accurate. The effort spent by the linguists is reduced, but the quality of the final delivery may be compromised. In summary, although progress continues in the area of predictive and actual quality measures, there’s still a lot more to achieve but progress continues.
3. Crowdsourcing translation: moving between expectations and reality
Crowd translation first appeared as a solution for startups or nonprofit organizations that needed multilingual content, but had limited localization budgets. Nowadays, crowdsourcing is generally being used in contexts of post-editing machine translation. It can substantially improve the quality of machine translated content, but it can also be challenging in terms of quality monitoring.
By inviting a “crowd”, that can be a small group of colleagues or clients, or even the entire online community, text strings can be translated and quality control is carried out by peer-to-peer review – anyone can leave a comment or correction.
The risk is that this all depends on the interest and will to collaborate of the crowd members, as well as not knowing if they’re qualified or not. The verdict is that although it’s on a good path, there’s still a long learning curve to overcome.
The lingering question: Where is technology taking us?
Content technologies keep evolving at the speed of light and machine translation is no exception.
In today’s business world, global communication takes place in real time. To keep up with this pace, automation is inevitable. Companies all over the world are turning to machine translation to keep up with the constant rise of content volumes and an increasing “global village”.
As with any new design, this technological evolution raises interesting questions and dilemmas. Where is content technology taking us? Can we agree on standard metrics for measuring machine translation quality and performance? Can we ensure equal access to data for everyone? How do we manage content integrity? Can machine translation quality really be assessed without human review? Are solutions for voice recognition keeping pace?
As a global content solutions partner, we’re excited to be part of this content revolution and continue to help our customers find the ideal technology mix and quality level to communicate effectively with their target audiences worldwide. One thing is for sure: expectations are high to see what the European Association for Machine Translation (EAMT) will bring to MT Summit 2020!
To continue similar conversations, join us this September 30 at TAUS in Dublin where I will join industry leaders to explore current trends in machine translation, post editing, quality evaluations and technology evolution, as well as their applications in real world scenarios.
About the author
Gráinne Maycock is VP Sales at Amplexor, based in Dublin, Ireland. With over 17 years of executive-level experience in the localization industry, she has worked with many of the world’s largest organizations across the IT, life sciences, consumer products, telecommunications, manufacturing and marketing verticals. With a passion for delivering measurable results, Gráinne helps customers worldwide to create and implement content programs that increase their global market share and revenue. Gráinne was a member of the LocWorld committee and is involved in helping with Brand2Global, a global marketing event. She is also a regular speaker on globalization and localization topics at industry events.