Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lmtinfo database needs to done incrementally #44

Open
teuben opened this issue Mar 6, 2023 · 1 comment
Open

lmtinfo database needs to done incrementally #44

teuben opened this issue Mar 6, 2023 · 1 comment
Labels
Basically Done all but marked as done

Comments

@teuben
Copy link
Contributor

teuben commented Mar 6, 2023

for small amounts of raw data in DATA_LMT the "lmtinfo.py build" commands is fast, less than a minute, but on unity all the data exists, plus unity is just slow. So we need a mode in which this can be run incrementally.

In one example on a cpu node it took 2 hours, in another case via slurm it ran out of time, as our timelimit was set to 4 hours.

@teuben
Copy link
Contributor Author

teuben commented Apr 5, 2023

basically done, though not automated:

  make new2 OBSNUM0=xxxxxx

where the file last.obsnum should store the last obsnum that was catalogued

@teuben teuben added the Basically Done all but marked as done label Apr 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Basically Done all but marked as done
Projects
None yet
Development

No branches or pull requests

1 participant