Pipeline Parallelism implementation. #496

Job	Run time
CodeQL Scan	3m 2s
Dependency Review	4s
Calculate Tests To Trigger	7s
Test / gsm8k_small_g3_tp1	1h 0m 43s
Test / gsm8k_small_g3_tp2	6m 39s
Test / gsm8k_small_g2_tp1	9m 34s
Test / gsm8k_small_g2_tp2	1m 3s
Test / gsm8k_large_g3_tp2	1h 0m 52s
Test / gsm8k_large_g2_tp4	9m 4s
Test / gsm8k_small_g3_tp1_fp8	6m 15s
Test / gsm8k_small_g3_tp2_fp8	4m 56s
Test / gsm8k_small_g3_tp1_mss	3m 3s
Test / gsm8k_small_g2_tp1_mss	3m 35s
Test / gsm8k_small_g3_tp2_mss	3m 10s
Test / gsm8k_small_g2_tp2_mss	7m 13s
Test / gsm8k_small_g2_tp1_spec_decode	6m 59s
Test / gsm8k_small_g2_tp1_mlp_spec_decode	8m 9s
Test / gsm8k_small_g2_tp1_medusa_spec_decode	6m 53s
Test / gsm8k_small_g2_tp1_eagle_spec_decode	6m 49s
Test / test_llama_lora	1m 3s
Test / test_multilora	1m 3s
Test / test_long_context	1m 2s
	3h 31m 18s

Provide feedback