Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate tuning runs #2675

Merged
merged 8 commits into from
Aug 13, 2024
Merged

Automate tuning runs #2675

merged 8 commits into from
Aug 13, 2024

Conversation

MohamedElgammal
Copy link
Contributor

This PR adds a script that automates setting up large runs to tune multiple parameters. It also helps with parsing the resultant data generating a single .xlsx file with the full parsed data, average results over the different seeds and geometrically average results over all the circuits. The PR also includes a README that explains how to used the new script.

…meters and efficiently parse the resulting results
@github-actions github-actions bot added the lang-python Python code label Aug 5, 2024
@vaughnbetz
Copy link
Contributor

Thanks Mohamed; this looks like a useful script. It fails python linting though. I think doing a python autoformat would fix a lot of the issues (seem to be minor).

vtr_flow/scripts/tuning_runs/control_runs.py has lint errors
************* Module control_runs
vtr_flow/scripts/tuning_runs/control_runs.py:29:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:42:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:45:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:49:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:53:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:55:0: C0301: Line too long (105/100) (line-too-long)
vtr_flow/scripts/tuning_runs/control_runs.py:58:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:61:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:63:0: C0301: Line too long (114/100) (line-too-long)
vtr_flow/scripts/tuning_runs/control_runs.py:65:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:69:0: C0301: Line too long (145/100) (line-too-long)
vtr_flow/scripts/tuning_runs/control_runs.py:71:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:73:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:78:28: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:79:0: C0301: Line too long (107/100) (line-too-long)
vtr_flow/scripts/tuning_runs/control_runs.py:90:70: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:96:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:99:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:102:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:105:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:110:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:114:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:115:0: C0301: Line too long (123/100) (line-too-long)
vtr_flow/scripts/tuning_runs/control_runs.py:118:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:122:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:128:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:139:0: C0301: Line too long (116/100) (line-too-long)
vtr_flow/scripts/tuning_runs/control_runs.py:140:0: C0301: Line too long (122/100) (line-too-long)
vtr_flow/scripts/tuning_runs/control_runs.py:141:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:144:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:175:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:178:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:182:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:185:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:188:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:192:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:194:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:197:0: C0303: Trailing whitespace (trailing-whitespace)
vtr_flow/scripts/tuning_runs/control_runs.py:204:0: C0305: Trailing newlines (trailing-newlines)
vtr_flow/scripts/tuning_runs/control_runs.py:1:0: C0114: Missing module docstring (missing-module-docstring)
vtr_flow/scripts/tuning_runs/control_runs.py:5:0: E0401: Unable to import 'pandas' (import-error)
vtr_flow/scripts/tuning_runs/control_runs.py:6:0: E0401: Unable to import 'numpy' (import-error)
vtr_flow/scripts/tuning_runs/control_runs.py:7:0: E0401: Unable to import 'scipy' (import-error)
vtr_flow/scripts/tuning_runs/control_runs.py:17:0: C0103: Constant name "keep_metrics_only" doesn't conform to UPPER_CASE naming style (invalid-name)
vtr_flow/scripts/tuning_runs/control_runs.py:21:0: C0116: Missing function or method docstring (missing-function-docstring)
vtr_flow/scripts/tuning_runs/control_runs.py:25:0: C0116: Missing function or method docstring (missing-function-docstring)
vtr_flow/scripts/tuning_runs/control_runs.py:25:26: W0621: Redefining name 'params_dict' from outer scope (line 10) (redefined-outer-name)
vtr_flow/scripts/tuning_runs/control_runs.py:36:0: C0116: Missing function or method docstring (missing-function-docstring)
vtr_flow/scripts/tuning_runs/control_runs.py:36:30: W0621: Redefining name 'params_dict' from outer scope (line 10) (redefined-outer-name)
vtr_flow/scripts/tuning_runs/control_runs.py:36:0: R0914: Too many local variables (18/15) (too-many-locals)
vtr_flow/scripts/tuning_runs/control_runs.py:76:10: W1309: Using an f-string that does not have any interpolated variables (f-string-without-interpolation)
vtr_flow/scripts/tuning_runs/control_runs.py:80:10: W1309: Using an f-string that does not have any interpolated variables (f-string-without-interpolation)
vtr_flow/scripts/tuning_runs/control_runs.py:83:10: W1309: Using an f-string that does not have any interpolated variables (f-string-without-interpolation)
vtr_flow/scripts/tuning_runs/control_runs.py:85:0: C0116: Missing function or method docstring (missing-function-docstring)
vtr_flow/scripts/tuning_runs/control_runs.py:100:0: C0116: Missing function or method docstring (missing-function-docstring)
vtr_flow/scripts/tuning_runs/control_runs.py:100:39: W0621: Redefining name 'params_dict' from outer scope (line 10) (redefined-outer-name)
vtr_flow/scripts/tuning_runs/control_runs.py:127:0: C0116: Missing function or method docstring (missing-function-docstring)
vtr_flow/scripts/tuning_runs/control_runs.py:140:62: C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary)
vtr_flow/scripts/tuning_runs/control_runs.py:155:0: C0116: Missing function or method docstring (missing-function-docstring)
vtr_flow/scripts/tuning_runs/control_runs.py:155:60: W0621: Redefining name 'params_dict' from outer scope (line 10) (redefined-outer-name)
vtr_flow/scripts/tuning_runs/control_runs.py:171:0: C0116: Missing function or method docstring (missing-function-docstring)
vtr_flow/scripts/noc/noc_benchmark_test.py passed

@MohamedElgammal
Copy link
Contributor Author

@vaughnbetz The linting issue is now fixed. The remaining failures are unrelated to the PR.

@vaughnbetz
Copy link
Contributor

Thanks!

@vaughnbetz vaughnbetz merged commit 21d0150 into master Aug 13, 2024
36 of 53 checks passed
@vaughnbetz vaughnbetz deleted the script_for_tuning_runs branch August 13, 2024 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lang-python Python code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants