how to increase reordering diversity of n-best list generated by Moses

[Saeed Farzi]

Dear All,


I am going to increase reordering diversity of n-best list generated by
Moses in order to apply my reordering model (a re-ranker system) for
improving translation quality.

In the first step due to low reordering diversity of the n-best list, i
need that the n-best list is included by different sentences which have
different reorderings.


For increasing the n-best list diversity, we generate 10000-best list and
then select 100-best of sentences with different reorderings.

For generating 10000-best list we use stack diversity and cube-pruning
options.
There are two disadvantages:

1) This task is very time consuming because of generating 10000-best in the
first step.

2) we loose lots of good translation candidates in terms of lexical choice.
It is caused that our reranker system could not find better translation
candidates in order to improve translation quality (BLEU score).


I am looking for new solution for increasing reordering diversity
without aforementioned problems.

Cheers

[Philipp Koehn]

Hi,

to obtain an n-best list with reordering diversity,
you could dump out the search graph and search for
reorderedly diverse translations according to a your
preferences.

-phi

Post by Saeed Farzi
Dear All,
I am going to increase reordering diversity of n-best list generated by
Moses in order to apply my reordering model (a re-ranker system) for
improving translation quality.
In the first step due to low reordering diversity of the n-best list, i
need that the n-best list is included by different sentences which have
different reorderings.
For increasing the n-best list diversity, we generate 10000-best list and
then select 100-best of sentences with different reorderings.
For generating 10000-best list we use stack diversity and cube-pruning
options.
1) This task is very time consuming because of generating 10000-best in
the first step.
2) we loose lots of good translation candidates in terms of
lexical choice. It is caused that our reranker system could not find better
translation candidates in order to improve translation quality (BLEU score).
I am looking for new solution for increasing reordering diversity
without aforementioned problems.
Cheers
--
S.Farzi, Ph.D. Student
Natural Language Processing Lab,
School of Electrical and Computer Eng.,
Tehran University
Tel: [phone]
_______________________________________________
Moses-support mailing list
http://mailman.mit.edu/mailman/listinfo/moses-support