Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible error in C# HumanEval test cases #13

Open
anmolagarwal999 opened this issue Sep 20, 2023 · 1 comment
Open

Possible error in C# HumanEval test cases #13

anmolagarwal999 opened this issue Sep 20, 2023 · 1 comment

Comments

@anmolagarwal999
Copy link

anmolagarwal999 commented Sep 20, 2023

It seems that a statement such as:

 var expected1 = null;

is not allowed in C# and throws an error of the form: Cannot assign <null> to an implicitly-typed local variable. It seems that this error is persistent across all versions of Dotnet (ref) . However, this statement seems to be used in several parts of the testing code such as: here and here.

Am I missing something ? Can someone look into this ?
\c @benathi @sk-g @hyandell

@mrward
Copy link

mrward commented Oct 10, 2023

Also ran into this problem. Also noticed some other C# code errors in the test cases.

  1. HumanEval_csharp12 uses 'var expected1 = null'
  2. HumanEval_csharp47 The test case does not seem to have been ported correctly from Python. The median of '-10,4,6,1000,10,20' is 8 not 15.
  3. HumanEval_csharp90 uses 'var expected3 = null'
  4. HumanEval_csharp114 uses new List {-9999999999999999} and the number is too big for an int
  5. HumanEval_csharp128 uses 'var expected4 = null'
  6. HumanEval_csharp137 uses 'var expected8 = null'
  7. HumanEval_csharp160 uses 'operator' as a method parameter name
  8. HumanEval_csharp162 uses 'var expected2 = null

The following HumanEval test cases are not ported from Python (which I am assuming is deliberate):

  • 32
  • 38
  • 50

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants