[Feature]: Compile warmup take too long

[Zjq9409]

🚀 The feature, motivation and pitch

INFO 08-26 07:31:47 habana_model_runner.py:1192] [Warmup][Prompt][1/56] batch_size:64 seq_len:1024 free_mem:13.93 GiB
INFO 08-26 07:32:25 habana_model_runner.py:1192] [Warmup][Prompt][2/56] batch_size:64 seq_len:896 free_mem:13.86 GiB
INFO 08-26 07:32:59 habana_model_runner.py:1192] [Warmup][Prompt][3/56] batch_size:64 seq_len:768 free_mem:13.86 GiB
INFO 08-26 07:33:27 habana_model_runner.py:1192] [Warmup][Prompt][4/56] batch_size:64 seq_len:640 free_mem:13.86 GiB
INFO 08-26 07:33:52 habana_model_runner.py:1192] [Warmup][Prompt][5/56] batch_size:32 seq_len:1024 free_mem:13.86 GiB
INFO 08-26 07:34:08 habana_model_runner.py:1192] [Warmup][Prompt][6/56] batch_size:64 seq_len:512 free_mem:13.86 GiB
INFO 08-26 07:34:25 habana_model_runner.py:1192] [Warmup][Prompt][7/56] batch_size:32 seq_len:896 free_mem:13.86 GiB
INFO 08-26 07:34:40 habana_model_runner.py:1192] [Warmup][Prompt][8/56] batch_size:32 seq_len:768 free_mem:13.86 GiB

Could it be compiled with multiple threads, if the graph compilation use CPU? During my compilation process, CPU utilization is very low.

Alternatives

No response

Additional context

No response

[mswiniarsk]

@Zjq9409 yes, we are planning to decrease warmup time in several directions:

• decrease compilation time, so that single graph takes less time
• more sophisticated way of selecting buckets, so that we are not compiling graphs that are not truly needed
• parallel compilation

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!