Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reenable writing index only #300

Merged
merged 1 commit into from
Dec 2, 2024
Merged

Reenable writing index only #300

merged 1 commit into from
Dec 2, 2024

Conversation

ASLeonard
Copy link
Contributor

Relatively minor change after some parameters changed. Not sure if indexing on the fly will be the preferred option going forward, but seems intuitive if you want to write the index that that is all you want to do. Also assumes if you have an index and still want to write an index, you want to overwrite it

@ekg
Copy link
Collaborator

ekg commented Nov 25, 2024

I was hoping we could do this via -W[FILE], --write-index=[FILE] build and save index to FILE. Does that not work?

@ekg
Copy link
Collaborator

ekg commented Nov 25, 2024

this only builds the index and saves to file

@ASLeonard
Copy link
Contributor Author

Using --write-index will write the index file but then continue on to the mapping stage. Building the index separately would allow something like the all2all_jobs.py script or some other allocation approach to do node-level parallelisation of the mappings once the index has been built a single time.

@ekg
Copy link
Collaborator

ekg commented Nov 30, 2024

Ok, then we should fix this. The behavior should be to write and exit without mapping. I would not want to map too or have an option to do that. We can load the same index.

@ekg ekg merged commit 9c15c7d into waveygang:main Dec 2, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants