Can't find the dataset lighteval/MATH-Hard in the huggingface

[Chenxi622]

when I run the code to get the results of the tasks in the open llm leaderboard v2, I can't find lighteval/MATH-Hard in the huggingface, how should I solve this problem?
Thanks！

[zxia545]

Have the same issue

[baberabb]

Hi! yeah, it doesn't seem to be up anymore. cc: @clefourrier @NathanHB

[stijn-zyphra]

Did one find a temporary solution for this with an exactly similar dataset? I reckon this dataset would probably be uploaded to HF multiple times, so forking lm_eval or monkey patching with the reference to that one would be feasible for now. How long does it usually take before these kind of problems are resolved? Thanks in advance.

[baberabb]

As it's the same dataset as MATH, but they only use level-5 problems, you could change the dataset_path here to EleutherAI/hendrycks_math and pipe a filter in the process_docs function (return dataset.filter(lambda x: x["level"] == "Level 5").map(...)), in the utils script (same folder).

Also add "level": "Level 5", field to all the samples in list_fewshot_samples

[stijn-zyphra]

Thank you very much, will fork it and implement a custom solution!

[Chenxi622]

Thank you for the answer！I will try it.