Bring back TorchBench AO dashboard

[huydhn]

After pytorch/benchmark#2561, TorchBench AO benchmark data is now available to query and we can finally use that dashboard again. If this proves useful, the next steps would be:

1. Resurrect ao benchmark on AWS A100 runner benchmark#2561 only brings back one example model for each suite (TorchBench, HF, TIMM). We need to add more.
2. TorchBench AO dashboard shares the code with TorchInductor dashboard. While the former has been migrated to the new benchmark database, the latter hasn't. I will need to do that and clean this up in a later PR.
3. Looking at the results on the dashboard, it seems that autoquant works, but not int8dynamic and int8weightonly. I'm not sure if they are still relevant, but if they are, ao team should know how to fix them (cc @jerryzh168). The run on TorchBench is at https://github.com/pytorch/benchmark/actions/workflows/torchao.yml

Testing

The two metrics speedup and abs execution time are now showing up https://torchci-git-fork-huydhn-ch-migrate-torchao-queries-fbopensource.vercel.app/benchmark/torchao?dashboard=torchao&startTime=Sun%2C%2015%20Dec%202024%2011%3A06%3A45%20GMT&stopTime=Sun%2C%2022%20Dec%202024%2011%3A06%3A45%20GMT&granularity=hour&mode=inference&dtype=autoquant&deviceName=cuda%20(a100)&lBranch=main&lCommit=07e6ef43fca2e95bc6cf59f97ba6251e618ef0e3&rBranch=main&rCommit=c03fa7c6c1bd03242a9de1fddb77a9c778106afd

[vercel]

@huydhn is attempting to deploy a commit to the Meta Open Source Team on Vercel.

A member of the Team first needs to authorize it.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
torchci	✅ Ready (Inspect)	Visit Preview	Dec 22, 2024 11:26am

[jerryzh168]

we don't need to track int8dynamic and int8weightonly right now I think

[jerryzh168]

we don't want to rely on default benchmark runner though, because of pytorch/benchmark#2527

[huydhn]

we don't want to rely on default benchmark runner though, because of pytorch/benchmark#2527

Yeah, I'm aware of the issue. This PR is about bringing back the UX for ao dashboard first because we need to do it before end of Dec (the upcoming deadline of our ClickHouse migration). So, I plan to decouple it from pytorch/benchmark#2527. If we end up not keeping this dashboard, we can drop this PR. But if we can fix pytorch/benchmark#2527, then it will be ready to land.

This is different than the progress on https://hud.pytorch.org/benchmark/llms?repoName=pytorch%2Fao where I'm adding the speedup summary there.