Skip to content

Latest commit

 

History

History
23 lines (19 loc) · 810 Bytes

README.md

File metadata and controls

23 lines (19 loc) · 810 Bytes

GSM8K-Consistency Benchmark

GSM8K-Consistency is a benchmark database for analyzing the consistency of Arithmetic Reasoning on GSM8K.

🚀 The dataset is available on 🤗 Hugging Face!

This is a math-problem-related semantics-preserving perturbation benchmark that can be very helpful for evaluating the consistency of arithmetic reasoning capability.

💻 Dataset Usage

Run the following command to load the data:

from datasets import load_dataset

dataset = load_dataset("shuyuej/GSM8K-Consistency")
dataset = dataset['train']
print(dataset)

Dataset Description:

Dataset({
    features: ['id', 'original_question', 'paraphrased_question', 'answer_detail', 'numerical_answer'],
    num_rows: 85225
})