Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Beginning of acceptance test framework #12

Merged
merged 10 commits into from
Oct 31, 2024
Merged

Beginning of acceptance test framework #12

merged 10 commits into from
Oct 31, 2024

Conversation

yasuiniko
Copy link
Collaborator

@yasuiniko yasuiniko commented Oct 25, 2024

This is the start of an acceptance test using mountaincar and softmaxAC. So far I have done:

  • Added tests/acceptance/softmaxAC_mc.py with a test that writes to a database using DefinitionPart and reads from it using ExperimentDefinition.
  • Added a unit test tests/test_generate_configurations.py, which tests the function in DefinitionPart.py.

@yasuiniko yasuiniko force-pushed the acceptance branch 2 times, most recently from 8857edd to 9984392 Compare October 26, 2024 00:59
@yasuiniko
Copy link
Collaborator Author

yasuiniko commented Oct 26, 2024

The tests are failing because Scheduler imports typing.Self, which was implemented in Python 3.11. The import is not from my code, it is from main. Should we stop testing python 3.10?

@panahiparham
Copy link
Owner

  1. split styling changes and tests
  2. maybe remove python 3.10 from testing?
  3. make sure local run actually runs everything

@yasuiniko
Copy link
Collaborator Author

yasuiniko commented Oct 30, 2024

@andnp we need your advice here. The Github workflow tests for this PR are failing because of two issues

  1. Scheduler imports typing.Self, which is only introduced with python 3.11. The workflow tests include python 3.10.
  2. The user's experiment.py cannot import ml_experiment since it is not pip-installed during the github workflow, and ml_experiment is not in sys.path. See for more details: Git workflow: can't import ml_experiment in a script that pytest runs via subprocess.run #20

I can fix both of these issues by editing .github/workflow/tests.yml, but I wanted to make sure you think this is a good idea first.

This makes pytest compatible with _utils.path.get_experiment_name()

Fixes: #18
@andnp
Copy link
Collaborator

andnp commented Oct 31, 2024

Yup, I think editing the workflow spec to solve both of these is a great strategy! I'm quite comfortable dropping 3.10 support

@yasuiniko
Copy link
Collaborator Author

yasuiniko commented Oct 31, 2024

Below tests are those on the main branch, the PR has an updated test.yaml and those pass when run locally through https://github.com/nektos/act. This fixes #20.

@panahiparham panahiparham merged commit 2971444 into main Oct 31, 2024
1 of 5 checks passed
@yasuiniko yasuiniko deleted the acceptance branch October 31, 2024 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants