[Feature] Parallelism Experiments on AIMO and LIMO

[zhaochenyang20]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

Can anyone help test @simon V’s branch? It’s pretty complete, but we’d like to run some parallel experiments

#3532

Feel free to submit a PR reporting the results of the parallel experiments, including std, var, etc. Thanks!

Related resources

No response

[tanzelin430]

Hi Chenyang, I’d love to test Simon’s new branch!
I now have 8 * 4090D GPUs, which don’t support P2P. Not sure if this disadvantage will have any significant impact on the results.
And in case that the pr has already been merged. Maybe we can test it directly on main branch?

[zhaochenyang20]

@tanzelin430 cool. You can ask Simon for help. I've told him @simveit

[zhaochenyang20]

8 * 4090 is enough. If you needs more, contact me please. @tanzelin430

[simveit]

Hello @tanzelin430 can you contact me in Slack channel or WeChat. I am Simon V on slack.

[simveit]

I opened this issue where I explain the experiment a little bit more detailed.
@zhaochenyang20 I think we can close this issue in favor of the one i wrote.
@tanzelin430 feel free to contact me if you need further help.

[zhaochenyang20]

@simveit Hope to see your collabration with @tanzelin430