Title:  Unsupervised Machine Translation

Non: Teleconference: https://eu.bbcollab.com/guest/b22b606d9ae74bc5b3e067821c897617
Faculty of informatics (UPV/EHU) Ada Lovelace room
Date: July 29, 2020, Wednesday,  11:00
Author: Mikel Artetxe Zurutuza 
Supervisors: Eneko Agirre & Gorka Labaka
Languages:  Basque (motivation, state of the art)  and English (second half, papers, conclusions, ~11:30…)

https://github.com/artetxem

https://github.com/artetxem

Abstract:

The advent of neural sequence-to-sequence models has led to impressive progress in machine translation, with large improvements in standard benchmarks and the first solid claims of human parity in certain settings. Nevertheless, existing systems require strong supervision in the form of parallel corpora, typically consisting of several million sentence pairs. Such a requirement greatly departs from the way in which humans acquire language, and poses a major practical problem for the vast majority of low-resource
language pairs.

The goal of this thesis is to remove the dependency on parallel data altogether, relying on nothing but monolingual corpora to train unsupervised machine translation systems. For that purpose, our approach first aligns separately trained word representations in
different languages based on their structural similarity, and uses them to initialize either a neural or a statistical machine translation system, which is further trained through back-translation.

Mikel Artetxe publications related to his PhD work:

•  Mikel Artetxe, Sebastian Ruder, Dani Yogatama, Gorka Labaka, Eneko Agirre (2020)
  A Call for More Rigor in Unsupervised Cross-lingual Learning
  Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
• Mikel Artetxe, Gorka Labaka, Eneko Agirre (2019)
  Bilingual Lexicon Induction through Unsupervised Machine Translation
  Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5002-5007.
• Mikel Artetxe, Gorka Labaka, Eneko Agirre (2019)
  An Effective Approach to Unsupervised Machine Translation
  Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 194-203.
• Mikel Artetxe, Gorka Labaka, Eneko Agirre (2018)
  Unsupervised Statistical Machine Translation
  In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3632–3642, Brussels, Belgium, October-November. Association for Computational Linguistics.
• Mikel Artetxe, Gorka Labaka, Eneko Agirre (2018)
  A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
  Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics
• Mikel Artetxe, Gorka Labaka, Eneko Agirre, Kyunghyun Cho (2018)
  Unsupervised Neural Machine Translation
  Sixth International Conference on Learning Representations (ICLR 2018)
• Mikel Artetxe, Gorka Labaka, Eneko Agirre (2018)
  Generalizing and Improving Bilingual Word Embedding Mappings with a Multi-Step Framework of Linear Transformations
  Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) pages 5012-5019.
• Mikel Artetxe, Gorka Labaka, Eneko Agirre (2017)
  Learning bilingual word embeddings with (almost) no bilingual data
  Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics
• Mikel Artetxe, Gorka Labaka, Eneko Agirre (2016)
  Learning principled bilingual mappings of word embeddings while preserving monolingual invariance
  Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2289–2294. Austin, Texas. ISBN: 978-1-945626-25-8
• 

Recently, at the beginging of June, this project succesfully organized in Denver, Colorado, the SSST-9 – Ninth Workshop on Syntax, Semantics and Structure in Statistical Translation collocated with NAACL 2015, June 4, 2015.

Yesterday, the results of this project were presented in a press conference with representatives of the Basque Government. Throughout the BerbaTek project we have created several language tools, resources and some demos to show the potential of the integration of language, voice and multimedia technologies, when it comes to creating applications for the areas that make up the languages industry, in other words, for translation, contents and teaching. Three demos were presented in the press conference:

• Automatic dubbing demo. The automatic dubbing of films is a difficult challenge for the moment (different voices, colloquial language, different speeds…), but for some types of documentaries (single speaker, voice-over, coordination of the lips not necessary or unimportant …) we’ve done a demo that performs satisfactorily. Given a documentary in Spanish and its transcription (which can be obtained automatically by means of any of the dictation programs for Spanish in the market), Vicomtech–IK4’s temporal alignment technology creates a subtitles file, a transcription with time marks for the beginning and end of each sentence. Then, the Matxin MT system, developed by the IXA group, automatically translates the subtitles into Basque, and Aholab’s text-to-speech technology obtains the synchronized voice. We have successfully applied this demo to the single-speaker sections of the television program Teknopolis produced by Elhuyar. This demo can be seen at work here.
• Semantic multimedia search engine for science and technology content. This search engine is based on WNTerm, an ontology specialized in science and technology wich was created by Elhuyar and IXA. It is a network where scientific and technological terms are semantically related to each other, with subclasses, synonyms, etc. A new augmented version will be presented next month.
  
• Personal teacher for language learning. For the field of education, we have created a demo of a personal tutor for language learning. The tutor is a 3D avatar developed by Vicomtech-IK4 that shows emotions, can speak Basque and can understand what is said in Basque, using Aholab‘s technology. The tutor assists us in various tasks: we can do grammar exercises (verb conjugation, word inflection…) and reading comprehension exercises (fill in gaps in a text, choosing from several options) that are created automatically from texts using technology from IXA; we can evaluate our pronunciation, with Aholab technology; or it helps us when writing texts, with inflection of words, writing of numbers or querying dictionaries, by means of technology from IXA and Elhuyar. By the moment this demo works in local mode, but it will beavailable online by next spring.

The pieces of news has been received by media today:

• BasqueResearch, EuropaPress, Eitb.com, Yahoo! Noticias, Sustatu, Zuzeu, Argia
• ETB2 (39:52 minute), ETB1 (31:40 minute), Berria TB
• Diario Vasco, Berria, Gara, Noticias de Gipuzkoa.
• Euskadi+innova.net, Langune.com, Zientzia.net, Norteko Ferrokarrilla,

Further information about this project can be found at Berbatek project’s website.

Speaker: Tegau Andrews (Bangor University, Wales)
Uned Technolegau Iaith  /  Language Technologies Unit
Prifysgol Bangor     /   Bangor University

When: November 2, Wednesday
Where: Room 3.2
Time: 15.00
Title: From terminology standardization systems to machine translation: An overview of Welsh language technologies

Abstract:

An endangered language will progress if its speakers can make use of electronic technology” so postulates Wales-based linguistics professor David Crystal (Language Death, 2000: 141). Welsh, spoken by 20.8% of the population of Wales (Census 2001), is classed a vulnerable language by UNESCO, yet it is the Welsh Government’s stated aim to make Wales a truly bilingual nation.

This talk will focus on the progress being made in developing language technologies for Welsh speakers. It will range over topics such as Welsh machine translation, computer-aided translation tools, text-to-speech technology, terminology portals and e-learning resources, and present an overview of the work being done at the Terminology and Language Technologies Unit at Bangor University. The aim of such work is to enable and encourage Welsh speakers to use electronic technology in their own language.

Speaker: Daniele Pighin
          NLPRG, TALP
          Technical University of Catalonia, UPC
Date: May 31, 2011
Time: 11:30
Where: Computer Science Faculty, Room 3.2

Title
   Automatic Projection of Semantic Structures:
      an Application to Pairwise Translation Ranking
 
Abstract
The ability to automatically assess the quality of translation
hypotheses is a key requirement towards the development of accurate and
dependable translation models. While it is largely agreed that proper
transfer of predicate-argument structures from source to target is a
very strong indicator of translation quality, especially in relation to
adequacy, the incorporation of this kind of information in the
Statistical Machine Translation (SMT) evaluation pipeline is still
limited to few and isolated cases.

We present a model for the inclusion of semantic role annotations in the
framework of confidence estimation for machine translation. The model
has several interesting properties:
   1) it only requires a linguistic processor on the (generally
well-formed) source side of the translation;
   2) it does not directly rely on properties of the translation model
(hence, it can be applied beyond phrase-based systems);
   3) it is inherently extendable to cope with different kinds of
sequential annotations, e.g., POS tags.
These features make it potentially appealing for system ranking,
translation re-ranking and user feedback evaluation. Preliminary
experiments in pairwise hypothesis ranking on five confidence estimation
benchmarks show that the model has the potential to capture salient
aspects of translation quality.
]]>
					
					https://www.ehu.eus/ehusfera/ixa/2011/05/27/daniele-pighin/feed/
			0
		
		
			
		
		
		https://www.ehu.eus/ehusfera/ixa/2011/05/11/lihmt-2011/
					https://www.ehu.eus/ehusfera/ixa/2011/05/11/lihmt-2011/#comments
		
		
		Wed, 11 May 2011 09:21:45 +0000
				
		
		
		
		http://www.ehu.eus/ehusfera/ixa/?p=665

					Ixa Group in collaboration with TALP Centre from Technical University of Catalonia is organizing a one-day workshop on Using Linguistic Information for Hybrid Machine Translation (LIHMT-2011).
 
The workshop will be held in Barcelona on Friday, November 18, 2011.
 
Paper submission deadline is September 9 2011. See: http://ixa2.si.ehu.es/lihmt2011/
 
This is part of the dissemination effort of [...]]]>
										Ixa Group in collaboration with TALP Centre from Technical University of Catalonia is organizing a one-day workshop on

Using Linguistic Information for Hybrid Machine Translation (LIHMT-2011).
The workshop will be held in Barcelona on Friday, November 18, 2011.
Paper submission deadline is September 9 2011.  See: http://ixa2.si.ehu.es/lihmt2011/
This is part of the dissemination effort of our OpenMT-2 project.
]]>
					
					https://www.ehu.eus/ehusfera/ixa/2011/05/11/lihmt-2011/feed/
			1
		
		
			
		
		
		https://www.ehu.eus/ehusfera/ixa/2011/05/09/marquez/
					https://www.ehu.eus/ehusfera/ixa/2011/05/09/marquez/#comments
		
		
		Mon, 09 May 2011 10:10:00 +0000
				
		
		
		http://www.ehu.eus/ehusfera/ixa/?p=587

					 Speaker: Lluís Màrquez NLPRG, TALP Technical University of Catalonia, UPC
 
 Date: May 10, 2011 Time: 15:30 Where: Computer Science Faculty, Room 3.2
 
 Automatic evaluation in Machine Translation: Towards combined linguistically-motivated measures
 
Automatic evaluation plays a very important role in the development and comparison of machine translation systems. In this talk we [...]]]>
										
 Speaker: Lluís Màrquez

          NLPRG, TALP

          Technical University of Catalonia, UPC
 Date: May 10, 2011 

 Time: 15:30

 Where: Computer Science Faculty, Room 3.2
 Automatic evaluation in Machine Translation:

   Towards combined linguistically-motivated measures
Automatic evaluation plays a very important role in the development and comparison of machine translation systems.  In this talk we will overview the current trend of using linguistically-guided evaluation measures based on several linguistic layers and their combination. Also, we will talk about confidence estimation measures, a particular subset of measures to assess output quality without the need of reference translations.  Finally, we will overview the role of evaluation measures within the FAUST European project (Feedback Analysis for User Adaptive Statistical Translation; http://www.faust-fp7.eu/),

focusing on the usage of user feedback to guide the combination of measures.
]]>
					
					https://www.ehu.eus/ehusfera/ixa/2011/05/09/marquez/feed/
			1