Enable federated XGBoost using bootstrap aggregation in Task Runner

[kta-intel]

This PR enables a TaskRunner-based federated XGBoost using the bootstrap aggregation

Specifically this PR:

• creates an xgb_higgs task runner workspace to train on the higgs dataset [ref] with all required code (i.e. src/taskrunner.py, src/dataloader.py ,plan/*.yaml, etc.
• adds a tasks_xgb.yaml to enable new FedBaggingXGBoost aggregation when running xgb training workloads
• adds delta_updates parameter to Aggregator in order to bypass delta updating (for deep learning models getting weight deltas makes sense since the model size should stay relatively consistent, for tree-based algorithms, this makes less sense because more trees are added over time)
  • delta_updates is set to true by default to preserve normal behavior. xgboost taskrunner explicitly sets it to false to bypass it
• introduces new loader_xgb.py as the backend / superclass to src/dataloader.py
• introduces new runner_xgb.py as the backend / superclass to src/taskrunner.py
• introduces new federated boostrap algorithm for xgboost in aggregation_function.fed_bagging which bags the latest trees to a global model, consistent with currently accept federated xgboost algorithms in the industry

[MasterSkepticista]

General question: Have you used specific formatters for yaml files?

[kta-intel]

I copied over the yaml from other workspaces as a template then ran bash shell/format.sh in the whole repo. Is there something additional that you recommend?

[teoparvanov]

Awesome work, thanks @kta-intel! I have a couple of questions and comments, but overall the PR looks in excellent shape for such a sizeable new feature.

PS: is there an easy way to add a CI job that covers at least the "happy path" of an XGBoost-based federation?

[kta-intel]

PS: is there an easy way to add a CI job that covers at least the "happy path" of an XGBoost-based federation?

Thanks for the review Teo! I'm not sure what you mean by this, would this just be a toy sanity check CI?

[teoparvanov]

PS: is there an easy way to add a CI job that covers at least the "happy path" of an XGBoost-based federation?

Thanks for the review Teo! I'm not sure what you mean by this, would this just be a toy sanity check CI?

Yes, I mean a very basic CI job that does an E2E run of the XGBoost workspace. Like here, but using xgb_higgs as the template, and with a reduced test matrix (f.e. just ubuntu + python3.10). However, if the effort for this turns out to be significant, you can also consider doing it in a separate PR.

Other than that, the PR is ready to be merged IMO. Thanks, @kta-intel !

[kta-intel]

Yes, I mean a very basic CI job that does an E2E run of the XGBoost workspace. Like here, but using the xgb_higgs as a template, and with a reduced test matrix (f.e. just ubuntu + python3.10). However, if the effort for this turns out to be significant, you can also consider doing it in a separate PR.

I see! This is a good idea, but lets save it as a separate PR. Thanks for the review and suggestion(s) @teoparvanov!