zle-eng

opus-2020-06-28.zip

• dataset: opus
• model: transformer
• source language(s): bel bel_Latn orv_Cyrl rue rus ukr
• target language(s): eng
• model: transformer
• pre-processing: normalization + SentencePiece (spm32k,spm32k)
• download: opus-2020-06-28.zip
• test set translations: opus-2020-06-28.test.txt
• test set scores: opus-2020-06-28.eval.txt

Benchmarks

testset	BLEU	chr-F
newstest2012-ruseng.rus.eng	29.5	0.565
newstest2013-ruseng.rus.eng	23.8	0.509
newstest2014-ruen-ruseng.rus.eng	26.1	0.547
newstest2015-enru-ruseng.rus.eng	25.0	0.523
newstest2016-enru-ruseng.rus.eng	24.5	0.521
newstest2017-enru-ruseng.rus.eng	27.1	0.540
newstest2018-enru-ruseng.rus.eng	23.8	0.520
newstest2019-ruen-ruseng.rus.eng	25.5	0.528
Tatoeba-test.bel-eng.bel.eng	35.6	0.541
Tatoeba-test.multi.eng	47.3	0.633
Tatoeba-test.orv-eng.orv.eng	4.9	0.213
Tatoeba-test.rue-eng.rue.eng	12.5	0.307
Tatoeba-test.rus-eng.rus.eng	49.9	0.654
Tatoeba-test.ukr-eng.ukr.eng	49.5	0.654

opus-2020-07-27.zip

• dataset: opus
• model: transformer
• source language(s): bel bel_Latn orv_Cyrl rue rus ukr
• target language(s): eng
• model: transformer
• pre-processing: normalization + SentencePiece (spm32k,spm32k)
• download: opus-2020-07-27.zip
• test set translations: opus-2020-07-27.test.txt
• test set scores: opus-2020-07-27.eval.txt

Benchmarks

testset	BLEU	chr-F
newstest2012-ruseng.rus.eng	29.4	0.564
newstest2013-ruseng.rus.eng	23.6	0.510
newstest2014-ruen-ruseng.rus.eng	26.2	0.548
newstest2015-enru-ruseng.rus.eng	25.1	0.525
newstest2016-enru-ruseng.rus.eng	24.6	0.524
newstest2017-enru-ruseng.rus.eng	27.5	0.545
newstest2018-enru-ruseng.rus.eng	24.1	0.523
newstest2019-ruen-ruseng.rus.eng	25.2	0.529
Tatoeba-test.bel-eng.bel.eng	35.4	0.543
Tatoeba-test.multi.eng	48.1	0.640
Tatoeba-test.orv-eng.orv.eng	7.5	0.207
Tatoeba-test.rue-eng.rue.eng	18.7	0.344
Tatoeba-test.rus-eng.rus.eng	50.6	0.660
Tatoeba-test.ukr-eng.ukr.eng	50.2	0.659

opus2m-2020-08-01.zip

• dataset: opus2m
• model: transformer
• source language(s): bel bel_Latn orv_Cyrl rue rus ukr
• target language(s): eng
• model: transformer
• pre-processing: normalization + SentencePiece (spm32k,spm32k)
• download: opus2m-2020-08-01.zip
• test set translations: opus2m-2020-08-01.test.txt
• test set scores: opus2m-2020-08-01.eval.txt

Benchmarks

testset	BLEU	chr-F
newstest2012-ruseng.rus.eng	31.1	0.579
newstest2013-ruseng.rus.eng	24.9	0.522
newstest2014-ruen-ruseng.rus.eng	27.9	0.563
newstest2015-enru-ruseng.rus.eng	26.8	0.541
newstest2016-enru-ruseng.rus.eng	25.8	0.535
newstest2017-enru-ruseng.rus.eng	29.1	0.561
newstest2018-enru-ruseng.rus.eng	25.4	0.537
newstest2019-ruen-ruseng.rus.eng	26.8	0.545
Tatoeba-test.bel-eng.bel.eng	38.3	0.569
Tatoeba-test.multi.eng	50.1	0.656
Tatoeba-test.orv-eng.orv.eng	6.9	0.217
Tatoeba-test.rue-eng.rue.eng	15.4	0.345
Tatoeba-test.rus-eng.rus.eng	52.5	0.674
Tatoeba-test.ukr-eng.ukr.eng	52.1	0.673

opus4m-2020-08-12.zip

• dataset: opus4m
• model: transformer
• source language(s): bel bel_Latn orv_Cyrl rue rus ukr
• target language(s): eng
• model: transformer
• pre-processing: normalization + SentencePiece (spm32k,spm32k)
• download: opus4m-2020-08-12.zip
• test set translations: opus4m-2020-08-12.test.txt
• test set scores: opus4m-2020-08-12.eval.txt

Benchmarks

testset	BLEU	chr-F
newstest2012-ruseng.rus.eng	32.0	0.585
newstest2013-ruseng.rus.eng	25.7	0.527
newstest2014-ruen-ruseng.rus.eng	28.8	0.570
newstest2015-enru-ruseng.rus.eng	27.5	0.547
newstest2016-enru-ruseng.rus.eng	26.8	0.543
newstest2017-enru-ruseng.rus.eng	30.3	0.568
newstest2018-enru-ruseng.rus.eng	26.3	0.542
newstest2019-ruen-ruseng.rus.eng	28.1	0.554
Tatoeba-test.bel-eng.bel.eng	38.7	0.573
Tatoeba-test.multi.eng	50.7	0.660
Tatoeba-test.orv-eng.orv.eng	7.7	0.227
Tatoeba-test.rue-eng.rue.eng	13.1	0.335
Tatoeba-test.rus-eng.rus.eng	53.4	0.682
Tatoeba-test.ukr-eng.ukr.eng	52.5	0.677

opus1m+bt-2021-05-02.zip

• dataset: opus1m+bt
• model: transformer-align
• source language(s): bel orv rue rus ukr
• target language(s): eng
• model: transformer-align
• pre-processing: normalization + SentencePiece (spm32k,spm32k)
• download: opus1m+bt-2021-05-02.zip
• test set translations: opus1m+bt-2021-05-02.test.txt
• test set scores: opus1m+bt-2021-05-02.eval.txt

Benchmarks

testset	BLEU	chr-F	#sent	#words	BP
newstest2012.rus-eng	31.2	0.580	3003	72812	0.980
newstest2013.rus-eng	24.9	0.522	3000	64505	0.994
newstest2014-ruen.rus-eng	27.7	0.562	3003	69190	0.995
newstest2015-enru.rus-eng	26.9	0.541	2818	64744	0.951
newstest2016-enru.rus-eng	25.6	0.534	2998	69278	0.983
newstest2017-enru.rus-eng	29.1	0.559	3001	69033	0.966
newstest2018-enru.rus-eng	25.1	0.533	3000	71723	0.977
newstest2019-ruen.rus-eng	26.4	0.542	2000	42875	0.972
Tatoeba-test.bel-eng	43.8	0.620	2500	18567	0.988
Tatoeba-test.bel_Latn-eng	2.1	0.118	3	26	1.000
Tatoeba-test.multi-eng	50.1	0.656	10000	69896	0.973
Tatoeba-test.orv-eng	7.2	0.211	322	2102	0.997
Tatoeba-test.rue-eng	20.7	0.356	120	697	0.905
Tatoeba-test.rus-eng	51.5	0.668	10000	72891	0.965
Tatoeba-test.ukr-eng	51.2	0.667	10000	66113	0.973
tico19-test.rus-eng	27.1	0.582	2100	56347	1.000

opusTCv20210807+bt_transformer-big_2022-03-03.zip

• dataset: opusTCv20210807+bt
• model: transformer-big
• source language(s): bel bel_Latn orv_Cyrl rue rus ukr
• target language(s): eng
• raw source language(s): bel orv rue rus ukr
• raw target language(s): eng
• model: transformer-big
• pre-processing: normalization + SentencePiece (spm32k,spm32k)
• download: opusTCv20210807+bt_transformer-big_2022-03-03.zip
• test set translations: opusTCv20210807+bt_transformer-big_2022-03-03.test.txt
• test set scores: opusTCv20210807+bt_transformer-big_2022-03-03.eval.txt

Benchmarks

testset	BLEU	chr-F	#sent	#words	BP
newstest2012.rus-eng	38.9	0.63447	3003	72812	0.989
newstest2013.rus-eng	30.9	0.57307	3000	64505	1.000
newstest2014-ruen.rus-eng	40.1	0.65401	3003	69190	1.000
newstest2015-enru.rus-eng	35.7	0.61337	2818	64744	0.972
newstest2016-enru.rus-eng	35.1	0.61136	2998	69278	1.000
newstest2017-enru.rus-eng	40.1	0.64859	3001	69033	0.986
newstest2018-enru.rus-eng	34.8	0.61156	3000	71723	0.993
newstest2019-ruen.rus-eng	41.2	0.65136	2000	42875	1.000
Tatoeba-test-v2021-08-07.bel-eng	47.3	0.64466	2500	18567	0.983
Tatoeba-test-v2021-08-07.bel_Latn-eng	1.9	0.11821	3	26	1.000
Tatoeba-test-v2021-08-07.multi-eng	55.6	0.70282	10000	72538	0.978
Tatoeba-test-v2021-08-07.orv-eng	14.5	0.28505	322	2102	0.907
Tatoeba-test-v2021-08-07.rue-eng	18.4	0.35729	120	697	0.841
Tatoeba-test-v2021-08-07.rus-eng	56.6	0.70931	19425	147851	0.971
Tatoeba-test-v2021-08-07.ukr-eng	55.8	0.70441	13127	88600	0.989
tico19-test.rus-eng	33.0	0.62918	2100	56347	1.000

opusTCv20210807+bt_transformer-big_2022-03-17.zip

• dataset: opusTCv20210807+bt
• model: transformer-big
• source language(s): bel bel_Latn orv_Cyrl rue rus ukr
• target language(s): eng
• raw source language(s): bel orv rue rus ukr
• raw target language(s): eng
• model: transformer-big
• pre-processing: normalization + SentencePiece (spm32k,spm32k)
• download: opusTCv20210807+bt_transformer-big_2022-03-17.zip
• test set translations: opusTCv20210807+bt_transformer-big_2022-03-17.test.txt
• test set scores: opusTCv20210807+bt_transformer-big_2022-03-17.eval.txt

Benchmarks

testset	BLEU	chr-F	#sent	#words	BP
newstest2012.rus-eng	39.2	0.63735	3003	72812	0.986
newstest2013.rus-eng	31.3	0.57641	3000	64505	1.000
newstest2014-ruen.rus-eng	40.5	0.65667	3003	69190	1.000
newstest2015-enru.rus-eng	36.0	0.61718	2818	64744	0.970
newstest2016-enru.rus-eng	35.7	0.61414	2998	69278	0.999
newstest2017-enru.rus-eng	40.8	0.65367	3001	69033	0.984
newstest2018-enru.rus-eng	35.1	0.61378	3000	71723	0.990
newstest2019-ruen.rus-eng	41.6	0.65468	2000	42875	1.000
Tatoeba-test-v2021-08-07.bel-eng	48.0	0.65139	2500	18567	0.982
Tatoeba-test-v2021-08-07.bel_Latn-eng	2.1	0.11733	3	26	1.000
Tatoeba-test-v2021-08-07.multi-eng	56.0	0.70546	10000	72538	0.977
Tatoeba-test-v2021-08-07.orv-eng	13.5	0.28846	322	2102	0.963
Tatoeba-test-v2021-08-07.rue-eng	16.4	0.34539	120	697	0.817
Tatoeba-test-v2021-08-07.rus-eng	57.2	0.71309	19425	147851	0.968
Tatoeba-test-v2021-08-07.ukr-eng	56.9	0.71102	10000	67638	0.986
tico19-test.rus-eng	33.4	0.63284	2100	56347	1.000