Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REQUEST] 提供获取实验数据的代码接口 #738

Open
RongqinChen opened this issue Nov 13, 2024 · 0 comments
Open

[REQUEST] 提供获取实验数据的代码接口 #738

RongqinChen opened this issue Nov 13, 2024 · 0 comments
Labels
💪 enhancement New feature or request

Comments

@RongqinChen
Copy link

RongqinChen commented Nov 13, 2024

🤩 Features description [Please make everyone to understand it]

提供获取实验数据的代码接口,让用户可以通过代码下载指定项目名称的实验数据,方便用户更加高效的实验数据获取以及后续的数据分析。

👍 What problem does this feature solve

有了这个代码接口后,用户无需再逐一访问各个项目的网页,手动点击下载实验数据,这大大节省了时间和精力。
这样的设计不仅提升了用户体验,也提高了工作效率,例如更方便的计算一个项目中多个实验的各项指标统计值(如平均值、方差等),并轻松对比不同项目之间的实验效果。

👾 What does the proposed API look like

参考wandb的接口:

import pandas as pd
import numpy as np
import wandb
api = wandb.Api()

# Project is specified by <entity/project-name>
runs = api.runs(project_naame)

summary_list, config_list, name_list = [], [], []
for run in runs:
    # .summary contains the output keys/values for metrics like accuracy.
    #  We call ._json_dict to omit large files
    summary_list.append(run.summary._json_dict)

    # .config contains the hyperparameters.
    #  We remove special values that start with _.
    config_list.append({k: v for k, v in run.config.items() if not k.startswith('_')})

    # .name is the human-readable name of the run.
    name_list.append(run.name)

runs_df = pd.DataFrame({
    "summary": summary_list,
    "config": config_list,
    "name": name_list
})

# print(runs_df)
print(*summary_list, sep='\n')

best_test_metric_list = [
    summary['final/best_test_metric']
    for summary in summary_list
]
print(f"{np.mean(best_test_metric_list):.5f} +- {np.std(best_test_metric_list):.5f}")

runs_df.to_csv(f"{project_naame}.csv")

🚑 Any additional [like screenshots]

@RongqinChen RongqinChen added the 💪 enhancement New feature or request label Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💪 enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant