[RMP] Potential New Examples and Example Tasks

[bschifferer]

This is a list of new example ideas and other example tasks, which are not all prioritized. If some are prioritized, we can slice them out into a smaller tickets + individual tickets per example/task.

Tasks:

• Standardize usage of tf memory allocation (+ add disclaimer)
• Cleaning NVTabular examples
• Use testbook for all notebook unit tests
• Capture metrics for ASV

WIP:

• Add an end-2-end POC example with multi-hot columns
• [Task] Example of serving LightFM in Merlin Systems NVIDIA-Merlin/models#138

Prioritized:

• How to deal with large models
• Multi-model data input (e.g. adding images, sound, text) (required pre-trained vectors)
• Benchmark Dataloaders : Merlin vs. native dataloaders
• Benchmark mult-GPU training: Scaling from 1 GPU, 2x GPUs, 4x GPUs, 8x GPUs

Non-Prioritized Potential Examples:

• Adding local predictions both for ranking and retrieval models
• Optimizing embeddings
• Use-case: Search (might part of pre-trained embeddings)
• Adding examples for different cloud environments
• Use-cases for different industries
• CPU-only examples (provide as documentation)
• POC (feature store, etc.) for large scale
• POC (feature store, etc.) on cloud

Done:

• [Task] Add XGBoost Serving Example NVIDIA-Merlin/models#136
• Sequential-based for Merlin Models (if possible with current Merlin Models)
• [Task] Add Implicit Training Example NVIDIA-Merlin/models#561
• [Task] Example of serving Implicit in Merlin Systems NVIDIA-Merlin/models#137
• [Task] Example of training LightFM in Merlin Models NVIDIA-Merlin/models#562
• Examples for RecSys challenges (previous and currently running)
• Hyperparameter Tuning
• -showing improvement overtime
• Adding new examples for NVTabular
• Using implicit algorithm (waiting for inference)
• Example for data-parallel training and impact (in progress library development)
• [Task] Add example for dataloaders for new dataloader repo NVIDIA-Merlin/dataloader#12
• Example for model-parallel training (in progress library development)
• Example Booking.com