A Generic Character Aligned Machine Transliteration System for Indic Languages

A Generic Character Aligned Machine Transliteration System for Indic Languages
Author: Nikhil Londhe
Publisher:
Total Pages: 32
Release: 2013
Genre:
ISBN:

A typical problem encountered in machine translation is the Out of Vocabulary (OOV) terms. These are usually names of places, people or technical terms that cannot be easily translated from one language to another or become obfuscated when translated. These end up as transliterated terms, i.e., a syllable or syllable group conversion from one language to another while trying to preserve the phonetic pronunciation. Although a large number of transliteration systems have been built over the years, they suffer from several problems. Firstly, any machine learning system is only as good as the underlying dataset used to train the system. For resource poor languages thus, either no such systems exist or perform extremely poorly. Secondly, most transliteration systems are over fitted to cater to the source language. However, with the proliferation of the Internet and the social media, language mixing is fairly common and most such systems fail if words derived from other languages are introduced. In this research, we aim to build better transliteration systems that can better model the language under consideration and incorporate additional features that can offset the over fitting problem described above. Also we explore how inherent language similarities can be used to bootstrap transliteration systems for resource poor languages. We explore how classical techniques in machine translation and information retrieval can be adapted to the problem in hand to build better and more robust systems.

Machine Translation and Transliteration involving Related, Low-resource Languages

Machine Translation and Transliteration involving Related, Low-resource Languages
Author: Anoop Kunchukuttan
Publisher: CRC Press
Total Pages: 215
Release: 2021-09-08
Genre: Computers
ISBN: 1000422410

Machine Translation and Transliteration involving Related, Low-resource Languages discusses an important aspect of natural language processing that has received lesser attention: translation and transliteration involving related languages in a low-resource setting. This is a very relevant real-world scenario for people living in neighbouring states/provinces/countries who speak similar languages and need to communicate with each other, but training data to build supporting MT systems is limited. The book discusses different characteristics of related languages with rich examples and draws connections between two problems: translation for related languages and transliteration. It shows how linguistic similarities can be utilized to learn MT systems for related languages with limited data. It comprehensively discusses the use of subword-level models and multilinguality to utilize these linguistic similarities. The second part of the book explores methods for machine transliteration involving related languages based on multilingual and unsupervised approaches. Through extensive experiments over a wide variety of languages, the efficacy of these methods is established. Features Novel methods for machine translation and transliteration between related languages, supported with experiments on a wide variety of languages. An overview of past literature on machine translation for related languages. A case study about machine translation for related languages between 10 major languages from India, which is one of the most linguistically diverse country in the world. The book presents important concepts and methods for machine translation involving related languages. In general, it serves as a good reference to NLP for related languages. It is intended for students, researchers and professionals interested in Machine Translation, Translation Studies, Multilingual Computing Machine and Natural Language Processing. It can be used as reference reading for courses in NLP and machine translation. Anoop Kunchukuttan is a Senior Applied Researcher at Microsoft India. His research spans various areas on multilingual and low-resource NLP. Pushpak Bhattacharyya is a Professor at the Department of Computer Science, IIT Bombay. His research areas are Natural Language Processing, Machine Learning and AI (NLP-ML-AI). Prof. Bhattacharyya has published more than 350 research papers in various areas of NLP.

Machine Translation and Transliteration Involving Related and Low-resource Languages

Machine Translation and Transliteration Involving Related and Low-resource Languages
Author: Anoop Kunchukuttan
Publisher: Chapman & Hall/CRC
Total Pages: 0
Release: 2021-08-12
Genre: Computers
ISBN: 9781003096771

Machine Translation and Transliteration involving Related, Low-resource Languages discusses an important aspect of natural language processing that has received lesser attention: translation and transliteration involving related languages in a low-resource setting. This is a very relevant real-world scenario for people living in neighbouring states/provinces/countries who speak similar languages and need to communicate with each other, but training data to build supporting MT systems is limited. The book discusses different characteristics of related languages with rich examples and draws connections between two problems: translation for related languages and transliteration. It shows how linguistic similarities can be utilized to learn MT systems for related languages with limited data. It comprehensively discusses the use of subword-level models and multilinguality to utilize these linguistic similarities. The second part of the book explores methods for machine transliteration involving related languages based on multilingual and unsupervised approaches. Through extensive experiments over a wide variety of languages, the efficacy of these methods is established. Features Novel methods for machine translation and transliteration between related languages, supported with experiments on a wide variety of languages. An overview of past literature on machine translation for related languages. A case study about machine translation for related languages between 10 major languages from India, which is one of the most linguistically diverse country in the world. The book presents important concepts and methods for machine translation involving related languages. In general, it serves as a good reference to NLP for related languages. It is intended for students, researchers and professionals interested in Machine Translation, Translation Studies, Multilingual Computing Machine and Natural Language Processing. It can be used as reference reading for courses in NLP and machine translation. Anoop Kunchukuttan is a Senior Applied Researcher at Microsoft India. His research spans various areas on multilingual and low-resource NLP. Pushpak Bhattacharyya is a Professor at the Department of Computer Science, IIT Bombay. His research areas are Natural Language Processing, Machine Learning and AI (NLP-ML-AI). Prof. Bhattacharyya has published more than 350 research papers in various areas of NLP.

Information Systems for Indian Languages

Information Systems for Indian Languages
Author: Chandan Singh
Publisher: Springer Science & Business Media
Total Pages: 331
Release: 2011-02-28
Genre: Computers
ISBN: 3642194028

This book constitutes the refereed proceedings of the International Conference on Information Systems for Indian Languages, ICISIL 2011, held in Patiala, India, in March 2011. The 63 revised papers presented were carefully reviewed and selected from 126 paper submissions (full papers as well as poster papers) and 25 demo submissions. The papers address all current aspects on localization, e-governance, Web content accessibility, search engine and information retrieval systems, online and offline OCR, handwriting recognition, machine translation and transliteration, and text-to-speech and speech recognition - all with a particular focus on Indic scripts and languages.

Computers and the Arabic Language

Computers and the Arabic Language
Author: Pierre A. MacKay
Publisher: Taylor & Francis
Total Pages: 266
Release: 1990
Genre: Computers
ISBN:

Based on papers from a Summer Session of the Arab School of Science and Technology, held near Damascus, July 1985.

Statistical Machine Translation

Statistical Machine Translation
Author: Philipp Koehn
Publisher: Cambridge University Press
Total Pages: 447
Release: 2010
Genre: Computers
ISBN: 0521874157

The dream of automatic language translation is now closer thanks to recent advances in the techniques that underpin statistical machine translation. This class-tested textbook from an active researcher in the field, provides a clear and careful introduction to the latest methods and explains how to build machine translation systems for any two languages. It introduces the subject's building blocks from linguistics and probability, then covers the major models for machine translation: word-based, phrase-based, and tree-based, as well as machine translation evaluation, language modeling, discriminative training and advanced methods to integrate linguistic annotation. The book also reports the latest research, presents the major outstanding challenges, and enables novices as well as experienced researchers to make novel contributions to this exciting area. Ideal for students at undergraduate and graduate level, or for anyone interested in the latest developments in machine translation.