Merge pull request #591 from liashahnazaryan/patch-75

Adding results to wmt23.md

applications/automatic-post-editing.md

@@ -42,6 +42,7 @@ When human post-edited translations are not available, synthetic post-editing da
 
 ### Evaluation
 Automatic post-editing systems can be evaluated like machine translation systems:
+
 -	Automatic, reference-based evaluation metrics, like [TER](/ter) or [BLEU](/bleu)
 -	Human evaluation, like direct assessment
 

events/wmt18.md

@@ -198,10 +198,10 @@ It was organised by [WMT](/wmt).
 
 ## Results
 
-Full results of the shared tasks: [*Findings of the 2018 Conference on Machine Translation (WMT18)*](https://aclanthology.org/W18-6401.pdf)
-
 ### News translation
 
+Full results of the shared task: [*Findings of the 2018 Conference on Machine Translation (WMT18)*](https://aclanthology.org/W18-6401.pdf)
+
 The results were determined with a monolingual [direct assessment](/human-evaluation-metrics#direct-assessment), the [average score and average z-score](/human-evaluation-metrics#average-score-and-average-z-score).
 
 #### → English

events/wmt19.md

@@ -205,10 +205,10 @@ It was organised by [WMT](/wmt).
 
 ## Results
 
-Full results of the shared tasks: [*Findings of the 2019 Conference on Machine Translation (WMT19)*](https://aclanthology.org/W19-5301.pdf)
-
 ### News translation
 
+Full results of the shared task: [*Findings of the 2019 Conference on Machine Translation (WMT19)*](https://aclanthology.org/W19-5301.pdf)
+
 The winner systems were listed according to their [average score and average z-score](/human-evaluation-metrics#average-score-and-average-z-score).
 
 #### → English

events/wmt20.md

@@ -233,10 +233,10 @@ It was organised by [WMT](/wmt).
 
 ## Results
 
-Full results of the shared tasks: [*Findings of the 2020 Conference on Machine Translation (WMT20)*](https://aclanthology.org/2020.wmt-1.1.pdf)
-
 ### News translation
 
+Full results of the shared task: [*Findings of the 2020 Conference on Machine Translation (WMT20)*](https://aclanthology.org/2020.wmt-1.1.pdf)
+
 The winner systems were listed according to their [average score and average z-score](/human-evaluation-metrics#average-score-and-average-z-score).
 
 #### → English

events/wmt21.md

@@ -226,10 +226,10 @@ It was organised by [WMT](/wmt).
 
 ## Results
 
-Full results of the shared tasks: [*Findings of the 2021 Conference on Machine Translation (WMT21)*](https://statmt.org/wmt21/pdf/2021.wmt-1.1.pdf)
-
 ### News translation
 
+Full results of the shared task: [*Findings of the 2021 Conference on Machine Translation (WMT21)*](https://statmt.org/wmt21/pdf/2021.wmt-1.1.pdf)
+
 The winner systems were listed according to their [average score and average z-score](/human-evaluation-metrics#average-score-and-average-z-score).
 
 #### → English

events/wmt22.md

@@ -125,11 +125,6 @@ Confirmed language pairs:
 > System papers must describe one or more shared task submissions. System paper submissions that we cannot link to a shared task submission will be rejected without review. System papers can overlap with other published work, and do not have to follow the double submission policy. There is no maximum length for system papers, but normally a short paper (4-6 pages) is appropriate. System papers should not be anonymised.
 
 
-## Poster format
-
-Posters must be presented in Gather.Town.
-
-
 ## Paper submission
 
 - Papers must be submitted [electronically](https://www.softconf.com/emnlp2022/WMT/).
@@ -322,10 +317,10 @@ Posters must be presented in Gather.Town.
 
 ## Results
 
-Full results of the shared tasks: [*Findings of the 2022 Conference on Machine Translation (WMT22)*](https://statmt.org/wmt22/pdf/2022.wmt-1.1.pdf)
-
 ### General task
 
+Full results of the shared task: [*Findings of the 2022 Conference on Machine Translation (WMT22)*](https://statmt.org/wmt22/pdf/2022.wmt-1.1.pdf)
+
 The winner systems were listed according to their [average score and average z-score](/human-evaluation-metrics#average-score-and-average-z-score).
 
 #### → English

events/wmt23.md

@@ -61,6 +61,41 @@ It was organised by [WMT](/wmt).
 
 In 2022, the *News* machine translation task was renamed the *General* machine translation task.
 
+## Scientific papers
+
+### Topics
+
+- Machine translation models (neural, statistical etc. )
+- Analysis of neural models
+- Using comparable corpora
+- Selection and preparation of data
+- Semi-supervised and unsupervised learning for machine translation, transfer learning
+- Multilingual machine translation
+- Incorporating linguistic information into machine translation
+- Machine translation inference
+- Manual and automatic methods for evaluating machine translation
+- Quality estimation
+
+### Research Papers
+
+> Research papers should describe original research corresponding to the categories listed above. Research papers that have been or will be submitted to other meetings or publications must indicate this at submission time, and must be withdrawn from the other venues if accepted and published at WMT 2022.
+
+> We will not accept for publication papers that overlap significantly in content or results with papers that have been or will be published elsewhere. It is acceptable to submit work that has been made available as a technical report (or similar, e.g. in arXiv) without citing it.
+
+> For the research track, papers should be anonymised, be between 6 and 10 pages in length (excluding references) and may include supplementary material.
+
+
+### System Papers
+
+> System papers must describe one or more shared task submissions. System paper submissions that we cannot link to a shared task submission will be rejected without review. System papers can overlap with other published work, and do not have to follow the double submission policy. There is no maximum length for system papers, but normally a short paper (4-6 pages) is appropriate. System papers should not be anonymised.
+
+
+## Paper submission
+
+- Papers must be submitted [electronically](https://www.softconf.com/emnlp2023/wmt/).
+- Research and system papers have the same deadlines.
+- Research and system papers should follow [EMNLP2022 formatting guidelines](https://2023.emnlp.org/calls/style-and-formatting/).
+
 ## Schedule
 
 ### Day 1
@@ -198,43 +233,34 @@ In 2022, the *News* machine translation task was renamed the *General* machine t
 | 17:15 - 17:30 | [**The Devil Is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation**](https://aclanthology.org/2023.wmt-1.100.pdf) <br>Patrick Fernandes, Daniel Deutsch, Mara Finkelstein, Parker Riley, André Martins, Graham Neubig, Ankush Garg, Jonathan Clark, Markus Freitag, Orhan Firat |
 
 
-## Scientific papers
+## Results
 
-### Topics
+### General task
 
-- Machine translation models (neural, statistical etc. )
-- Analysis of neural models
-- Using comparable corpora
-- Selection and preparation of data
-- Semi-supervised and unsupervised learning for machine translation, transfer learning
-- Multilingual machine translation
-- Incorporating linguistic information into machine translation
-- Machine translation inference
-- Manual and automatic methods for evaluating machine translation
-- Quality estimation
+Full results of the shared task: [*Findings of the 2023 Conference on Machine Translation (WMT23): LLMs Are Here but Not Quite There Yet*](https://aclanthology.org/2023.wmt-1.1.pdf)
 
-### Research Papers
-
-> Research papers should describe original research corresponding to the categories listed above. Research papers that have been or will be submitted to other meetings or publications must indicate this at submission time, and must be withdrawn from the other venues if accepted and published at WMT 2022.
-
-> We will not accept for publication papers that overlap significantly in content or results with papers that have been or will be published elsewhere. It is acceptable to submit work that has been made available as a technical report (or similar, e.g. in arXiv) without citing it.
-
-> For the research track, papers should be anonymised, be between 6 and 10 pages in length (excluding references) and may include supplementary material.
+The winner systems were listed according to their [average score](/human-evaluation-metrics#average-score-and-average-z-score).
+The results were determined with a bilingual direct assessment with [scalar quality metric](/human-evaluation-metrics#sqm) (SQM) with document context.
 
+#### → English
 
-### System Papers
+| Language pair | System | Average score |
+| --- | --- | --- |
+| German → | `GPT4-5shot` | 90.3 |
+| Chinese → | `Lan-BridgeMT`| 82.9 |
+| Japanese → | `GPT4-5shot` | 81.3 |
 
-> System papers must describe one or more shared task submissions. System paper submissions that we cannot link to a shared task submission will be rejected without review. System papers can overlap with other published work, and do not have to follow the double submission policy. There is no maximum length for system papers, but normally a short paper (4-6 pages) is appropriate. System papers should not be anonymised.
+#### English →
 
+| Language pair | System | Average score |
+| --- | --- | --- |
+| → German | `GPT4-5shot` | 89.0 |
+| → Czech | `ONLINE-W` | 84.1 |
+| → Chinese | `Yishu`| 82.2 |
+| → Japanese | `GPT4-5shot` | 79.5  |
 
-## Poster format
+#### Czech → Ukrainian
 
-- System description papers will be presentated as posters.
-- Poster panels are 3.28 foot (1 meter wide) x 8.20 foot (2.5 meter tall). Portrait orientation is suggested.
-
-
-## Paper submission
-
-- Papers must be submitted [electronically](https://www.softconf.com/emnlp2023/wmt/).
-- Research and system papers have the same deadlines.
-- Research and system papers should follow [EMNLP2022 formatting guidelines](https://2023.emnlp.org/calls/style-and-formatting/).
+| Language pair | System | Average score |
+| --- | --- | --- |
+| Czech → Ukrainian | `ONLINE-B` | 83.7 |