-
-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Devdocs offliner, category and warehouse path #1020
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1020 +/- ##
==========================================
+ Coverage 88.29% 88.34% +0.05%
==========================================
Files 96 97 +1
Lines 5389 5413 +24
==========================================
+ Hits 4758 4782 +24
Misses 631 631 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Category and warehouse paths require a decision. I haven't heard we're creating either for devdocs and so until then this should be other
for both I believe.
Excuse my questions if some doesn't make sense ; I haven't used the scraper yet.
Supportive of a dedicated Category and storage path. |
@rgaudin do not hesitate to ask for live call to settle on open discussions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think everything's settled but you did not re-request… approved anyway
Do you have a timestamped attorney-certified copy of that? 🙃 |
Rationale
Add new devdocs offliner, including new category (since we do not yet know which category list we want to use) and new warehouse path.
Also add new platform since we do not want to run more than one task per worker (but without global limit on the farm). See setting for our prod in kiwix/operations#263
Nota: while we have not yet released a 0.1, this is coming soon and this code should be ok to deploy since anyway we already have a
dev
Docker image at https://github.com/openzim/devdocs/pkgs/container/devdocs