Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Baseline for SGLang Benchmark Test #602

Merged
merged 48 commits into from
Dec 4, 2024
Merged

Add Baseline for SGLang Benchmark Test #602

Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
3c21be0
Add benchmark using sglang server,
stbaione Nov 22, 2024
31398a5
Fix import path in `shortfin_benchmark_test`,
stbaione Nov 22, 2024
4d0323f
Merge branch 'main' into sgl-benchmark-add-baseline
stbaione Nov 22, 2024
fc78284
Change `ci-sglang-benchmark/integration` to use `mi300x-4`,
stbaione Nov 22, 2024
0909e8f
Fix github runner label
stbaione Nov 22, 2024
d7cc539
Add installation steps, since test does require some functionality fr…
stbaione Nov 22, 2024
cf16e54
Fix typo in model names
stbaione Nov 22, 2024
86058b8
Add container name,
stbaione Nov 22, 2024
acbedb0
Temporarily remove `--rm` to try and obtain container logs after failure
stbaione Nov 22, 2024
34c8410
Remove quotes around HF_TOKEN
stbaione Nov 22, 2024
0d5574d
Try using env var for HF_SECRET
stbaione Nov 23, 2024
c9f4d33
Move secrets.HF_TOKEN back to command
stbaione Nov 25, 2024
4fa094c
Add temporary command to see if HF_TOKEN is being set properly
stbaione Nov 25, 2024
c33ef75
Add back command to rm container once stopped
stbaione Nov 25, 2024
6986765
Merge branch 'main' of https://github.com/nod-ai/shark-ai into users/…
stbaione Nov 25, 2024
fea2655
Allow for full e2e verification
stbaione Nov 25, 2024
3641445
Update hash for pip cache in benchmark and integration tests
stbaione Nov 25, 2024
d82d9df
Remove version pinning for `iree-base-compiler` and `iree-base-runtime`
stbaione Nov 25, 2024
e843281
Add `--pre` to iree installations in SGLang tests
stbaione Nov 26, 2024
7fe76d2
Merge branch 'main' into users/stbaione/sgl-benchmark-add-baseline
stbaione Nov 26, 2024
ea65936
Merge branch 'main' into users/stbaione/sgl-benchmark-add-baseline
stbaione Dec 2, 2024
01da13c
Slightly lower threshold in integration tests, to allow still valid, …
stbaione Dec 2, 2024
ed37ef1
Fix `publish_dir` in `Deploy to Github Pages` step
stbaione Dec 2, 2024
d1e434f
Merge branch 'main' into users/stbaione/sgl-benchmark-add-baseline
stbaione Dec 2, 2024
d29c7bb
Remove unneeded deps for SGLang benchmark,
stbaione Dec 2, 2024
4529e09
Comment out `needs` line for CI validation
stbaione Dec 2, 2024
09e0fb6
Remove temporary disablements,
stbaione Dec 2, 2024
422729f
Remove `Get Current Date` step in shortfin benchmark job
stbaione Dec 2, 2024
969b608
Add `README` description to top of CI file
stbaione Dec 2, 2024
f67e399
Add job to merge html reports from both benchmark jobs and upload to …
stbaione Dec 3, 2024
6909edc
Fix upload/download paths
stbaione Dec 3, 2024
aa35176
Split download into two steps
stbaione Dec 3, 2024
9578acc
Ensure all html files are in same dir
stbaione Dec 3, 2024
b9b9ea5
Remove PR trigger
stbaione Dec 3, 2024
526194f
Remove `sharktank` installation from SGLang benchmark,
stbaione Dec 3, 2024
57babdf
Use hf to download tokenizer in `sglang_benchmark_test`
stbaione Dec 3, 2024
8f7f0fb
Small cleanup of sglang ci deps section
stbaione Dec 3, 2024
305c4b0
Make shortfin/sglang benchmark such that they still run sequentially,…
stbaione Dec 3, 2024
d78ab73
Remove dep on shortfin benchmark in sgl benchmark,
stbaione Dec 3, 2024
29a8221
Make sure `merge_and_upload_reports` waits for prior jobs to finish
stbaione Dec 3, 2024
35960e0
Merge branch 'main' into users/stbaione/sgl-benchmark-add-baseline
stbaione Dec 3, 2024
7efecb8
Move code checkout to first step in `benchmark_sglang`
stbaione Dec 3, 2024
d28bf01
Merge branch 'users/stbaione/sgl-benchmark-add-baseline' of https://g…
stbaione Dec 3, 2024
1e90573
Remove PR trigger
stbaione Dec 3, 2024
3f6564f
Merge branch 'main' into users/stbaione/sgl-benchmark-add-baseline
stbaione Dec 4, 2024
b1ec485
Repin `iree-base-compiler` and `iree-base-runtime` due to `abort` issue
stbaione Dec 4, 2024
56d3a5c
Merge branch 'main' into users/stbaione/sgl-benchmark-add-baseline
stbaione Dec 4, 2024
5d406c8
Merge branch 'main' into users/stbaione/sgl-benchmark-add-baseline
stbaione Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'main' into users/stbaione/sgl-benchmark-add-baseline
  • Loading branch information
@stbaione
stbaione authored Nov 26, 2024
commit 7fe76d26947b5bd56a542f4ede989f59e28a9798

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.