Skip to content

Latest commit

 

History

History
53 lines (35 loc) · 1.82 KB

README.md

File metadata and controls

53 lines (35 loc) · 1.82 KB
Raw

Arabic EXAMS

Paper

EXAMS: a resource specialized in multilingual high school exam questions. The original paper EXAMS

The Arabic EXAMS dataset includes five subjects

  • Islamic studies
  • Biology
  • Physics
  • Science
  • Social

The original dataset EXAMS-QA

EXAMS is a benchmark dataset for cross-lingual and multilingual question answering for high school examinations. With 24,000 high-quality high school exam questions in 16 languages, covering 8 language families and 24 school subjects from Natural Sciences and Social Sciences, among others. EXAMS offers unique fine-grained evaluation framework across multiple languages and subjects

Homepage for Arabic EXAMS: EXAMS Arabic Homepage

Citation

Groups and Tasks

Groups

  • EXAMS Arabic: include IslamicStudies, Biology, Science, Physics, Social.

Tasks

The following tasks evaluate subjects in Arabic EXAMS dataset using loglikelihood-based multiple-choice scoring:

  • aexams_IslamicStudies
  • aexams_Biology
  • aexams_Science
  • aexams_Physics
  • aexams_Social

Checklist

  • Is the task an existing benchmark in the literature?
    • Have you referenced the original paper that introduced the task?
    • If yes, does the original paper provide a reference implementation?
      • Yes, original implementation contributed by author of the benchmark

If other tasks on this dataset are already supported:

  • Is the "Main" variant of this task clearly denoted?
  • Have you provided a short sentence in a README on what each new variant adds / evaluates?
  • Have you noted which, if any, published evaluation setups are matched by this variant?