[Feature]: support pipeline parallelism inference in vllm

[Zjq9409]

🚀 The feature, motivation and pitch

warnings.warn(
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 168, in <module>
    engine = AsyncLLMEngine.from_engine_args(
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/async_llm_engine.py", line 346, in from_engine_args
    engine_config = engine_args.create_engine_config()
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/arg_utils.py", line 549, in create_engine_config
    parallel_config = ParallelConfig(
  File "/usr/local/lib/python3.10/dist-packages/vllm/config.py", line 556, in __init__
    self._verify_args()
  File "/usr/local/lib/python3.10/dist-packages/vllm/config.py", line 560, in _verify_args
    raise NotImplementedError(
NotImplementedError: Pipeline parallelism is not supported yet.

Is pipeline parallelism in our roadmap?

Alternatives

No response

Additional context

No response

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!