[WIP] optimize infer_auto_device_map for multi-GPU allocation#3321

Draft

Nech-C wants to merge 2 commits intohuggingface:mainfrom Nech-C:feature/infer-auto-device-map-multi-gpu-alloc-optim

+82-21

Commits on Jan 2, 2025

feat: allow infer_auto_device_map to avoid unnecessary memory reservation
Nech-C
committed
Merge branch 'main' into feature/infer-auto-device-map-multi-gpu-alloc-optim
Nech-C
committed