Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some simple graphs characterising a workflow ... #310

Open
DavidPCoster opened this issue Feb 25, 2025 · 5 comments
Open

Some simple graphs characterising a workflow ... #310

DavidPCoster opened this issue Feb 25, 2025 · 5 comments

Comments

@DavidPCoster
Copy link
Contributor

I don't know if there is any interest in producing simple graphs characterising a workflow (simpler than those already provided). As an example

Image

which also produces a text version

compute.txt

The analysis script does this for compute, transfer and wait.

Image

Image

If there is interest, I could share my code -- but it is really so simple that anybody could reproduce it ...

@LourensVeen
Copy link
Contributor

I'm not sure how far to go with this actually. MUSCLE3 has some built-in graphs, and a Python API for loading profiling data and preprocessing it into easy-to-visualise data. I guess we can keep adding different kinds of visualisations, but eventually it would become a plotting library 😄.

Actually, maybe this would make for a nice example for the documentation?

@DavidPCoster
Copy link
Contributor Author

I find those graphs easier for getting a quick understanding than

Image
Image
Image
Image

@LourensVeen
Copy link
Contributor

Point taken 😄. Although that last plot comes close, it just has the profiling bug still in it that causes init to look like it's using a lot of CPU, and perhaps it should be sorted by amount of compute. Pie charts are also not the most accurate way of visualising data

The other issue is that a MUSCLE3 simulation is a parallel system, so just looking at the amount of time spent computing could also be misleading, as there could be a component that computes a lot while never actually being the long pole in the tent. So then it depends on whether you're trying to optimise runtime or resource usage.

@LourensVeen
Copy link
Contributor

Thinking about this more, which question are you trying to answer with these graphs? I think we need to get that clear to see if we need to improve the existing charts, add new ones, or something else.

@LourensVeen
Copy link
Contributor

And one last thought for today: how would we do this with hierarchical models? Maybe something like what graphical disk usage tools do? I seem to recall one that used a kind of 2D spatial tree, but searching the web a bit it seems that most of them are using pie charts now. As I recall it looked something like the first graph in this article on HPC machine market share.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants