Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Using AI to generate unit test code and improve memory leak? #1905

Open
SamYuan1990 opened this issue Jan 7, 2025 · 2 comments
Open
Assignees

Comments

@SamYuan1990
Copy link
Collaborator

Learn from kepler-metal-ci's report analysis, I want to proposal an new approach for us to improve our test code coverage and fixing memory leaks.
For now, we have our test coverage in report by golang command as ... --cover and memory leak detection as go build -xxx, which both are plain text result.
so
if we make a python script, reading those report, and

for unit test code generation.

filter by unit test result
- if this package is not on ban list(for example, version package do not need to test?)
- if some package's test coverage lower than a threshold.
- we send files inside this package to openAI to generate test result.
- we recevie the test code generate by openAI
- we rerun the test coverage with test files, if it improve test coverage for the package.
- create a new PR.

for memory leak detection

for each PR, filter by memory leak result
- if the issue is on ban list(for example, string or big slice/array moved to heap)
- if a new memory leak happen
- send the specific leak to openAI to see if this can be fixed or not?
- if new memory leak been added or the memory leak can be fixed as AI's suggestion.
- leave comments in PR for double check/manual review.
@sunya-ch
Copy link
Collaborator

sunya-ch commented Jan 7, 2025

We do have a way to enable pprof in Kepler: https://github.com/sustainable-computing-io/kepler/blob/main/doc/dev/README.md#profiling-kepler.
I think we can utilize it to spot the memory leak.

@SamYuan1990
Copy link
Collaborator Author

SamYuan1990 commented Jan 7, 2025

We do have a way to enable pprof in Kepler: https://github.com/sustainable-computing-io/kepler/blob/main/doc/dev/README.md#profiling-kepler. I think we can utilize it to spot the memory leak.

IMO, golang has two approaches for memory enhancement, one is pprof, and another one is escapes detect like https://github.com/sustainable-computing-io/kepler/actions/runs/12637476502/job/35211752748

maybe, we can enable pprof in benchmark testing(after we have unit test and benchmark testing, here means golang language level) and we can analysis pprof result as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants