Some examples for improving morphological disambiguation

[e-budur]

Hi,

Let me take first this opportunity to thank you all for your efforts in this projects.

I would like to give you some sample cases that might be useful when improving the disambiguation module even better.

Here is pom.txt file and DisambiguateSentences.java test file that I have tested the sample cases below.

Example 1: aracım arızalandı.
I|11:54:45.682|Sentence = aracım arızalandı.
I|11:54:45.682|Sentence word analysis result:
I|11:54:45.682|Word = aracım
I|11:54:45.682|[araç:Noun] arac:Noun+A3sg+ım:P1sg
I|11:54:45.682|[Arac:Noun,Prop] arac:Noun+A3sg+ım:P1sg
I|11:54:45.682|[aracı:Noun] aracı:Noun+A3sg+m:P1sg
I|11:54:45.682|[ara:Adj] ara:Adj|cı:Agt→Noun+A3sg+m:P1sg
I|11:54:45.682|[aracı:Adj] aracı:Adj|Zero→Noun+A3sg+m:P1sg
I|11:54:45.682|[araç:Noun] arac:Noun+A3sg|Zero→Verb+Pres+ım:A1sg
I|11:54:45.682|[Arac:Noun,Prop] arac:Noun+A3sg|Zero→Verb+Pres+ım:A1sg
I|11:54:45.682|[ara:Noun] ara:Noun+A3sg|cı:Agt→Noun+A3sg+m:P1sg
I|11:54:45.682|Word = arızalandı
I|11:54:45.682|[arızalanmak:Verb] arızalan:Verb+dı:Past+A3sg
I|11:54:45.682|[arıza:Noun] arıza:Noun+A3sg|lan:Acquire→Verb+dı:Past+A3sg
I|11:54:45.682|Word = .
I|11:54:45.682|[.:Punc] .:Punc
I|11:54:45.697|
After ambiguity resolution :
I|11:54:45.697|[araç:Noun] arac:Noun+A3sg+ım:P1sg
I|11:54:45.697|[arızalanmak:Verb] arızalan:Verb+dı:Past+A3sg
I|11:54:45.697|[.:Punc] .:Punc

Example 2: atm kurulması.
I|17:16:13.015|Sentence = atm kurulması.
I|17:16:13.019|Sentence word analysis result:
I|17:16:13.019|Word = atm
I|17:16:13.023|[Atm:Noun,Abbrv] atm:Noun+A3sg
I|17:16:13.023|Word = kurulması
I|17:16:13.023|[kurmak:Verb] kur:Verb|ul:Pass→Verb|ma:Inf2→Noun+A3sg+sı:P3sg
I|17:16:13.023|Word = .
I|17:16:13.023|[.:Punc] .:Punc
I|17:16:13.029|
After ambiguity resolution :
I|17:16:13.030|[Atm:Noun,Abbrv] atm:Noun+A3sg
I|17:16:13.030|[kurmak:Verb] kur:Verb|ul:Pass→Verb|ma:Inf2→Noun+A3sg+sı:P3sg
I|17:16:13.030|[.:Punc] .:Punc

Example 3: atm kurulumu.
I|17:10:15.499|Sentence = atm kurulumu.
I|17:10:15.504|Sentence word analysis result:
I|17:10:15.504|Word = atm
I|17:10:15.508|[Atm:Noun,Abbrv] atm:Noun+A3sg
I|17:10:15.509|Word = kurulumu
I|17:10:15.509|[kurul:Noun] kurul:Noun+A3sg+um:P1sg+u:Acc
I|17:10:15.509|[Kurulu:Noun,Prop] kurulu:Noun+A3sg+m:P1sg+u:Acc
I|17:10:15.509|[kurulum:Noun] kurulum:Noun+A3sg+u:Acc
I|17:10:15.509|[kurulum:Noun] kurulum:Noun+A3sg+u:P3sg
I|17:10:15.509|[kuru:Noun] kuru:Noun+A3sg|lu:With→Adj|Zero→Noun+A3sg+m:P1sg+u:Acc
I|17:10:15.509|[Kuru:Noun,Prop] kuru:Noun+A3sg|lu:With→Adj|Zero→Noun+A3sg+m:P1sg+u:Acc
I|17:10:15.509|[kuru:Adj] kuru:Adj|Zero→Noun+A3sg|lu:With→Adj|Zero→Noun+A3sg+m:P1sg+u:Acc
I|17:10:15.509|Word = .
I|17:10:15.509|[.:Punc] .:Punc
I|17:10:15.518|
After ambiguity resolution :
I|17:10:15.520|[Atm:Noun,Abbrv] atm:Noun+A3sg
I|17:10:15.520|[Kurulu:Noun,Prop] kurulu:Noun+A3sg+m:P1sg+u:Acc
I|17:10:15.520|[.:Punc] .:Punc

The sentence in the first example above couldn't be disambiguated in the previous versions (i.e. 0.9.3) but it is disambiguated pretty well in the current version (0.15.0). So, this example indicates that Averaged Perceptron that is used in the current version outperforms the previously adopted Markov Model approach.

However, the second and third examples, which were not disambiguated well also in the previous versions, show that there is still room for improvement in the disambiguation module.

I want to involve in this project when I have an appropriate time period. I will be glad if I can resolve this issue at that time. Yet, I wanted to write down these cases here, in case you figure out the root cause and implement a solution for it earlier.

Thanks,

Emrah

[ahmetaa]

Thanks, we are aware that disambiguation mechanism is not yet performing well. I will have a look at the examples before releasing 0.16.0.