Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature request] Integrating pod logs with SLS #300

Open
llinvokerl opened this issue Nov 22, 2023 · 1 comment
Open

[feature request] Integrating pod logs with SLS #300

llinvokerl opened this issue Nov 22, 2023 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@llinvokerl
Copy link

llinvokerl commented Nov 22, 2023

What would you like to be added:
I'd like to propose a new feature: integrating pod logs with Simple Logging Service (SLS) when configured by the user.

Here's the rationale:
In my current workflow, after a job is completed in kubedl, the associated pods are marked as 'Completed' but not deleted. This can lead to a scenario where large log files eventually fill up the disk space, which seems suboptimal.

My question is, given that system events can be stored through SLS configuration, why are pod logs not recorded? Capturing pod logs would allow for the deletion of pods post-job completion, addressing the potential disk space issue. Moreover, it appears that with SLS configured, users might lose access to any pod logs, which could be problematic. (I haven't verified this myself, but that's the impression I get from the code.)
image

I envision the pod log recording process as follows:

step1: When a pod status changes to 'Completed', the kubedl manager should immediately collect and write logs to SLS, record the pod details in a MySQL table, and then delete the pod.
step2: For log viewing in the console: 1. If the pod is running, fetch logs via apiserver; 2. If the pod is deleted (completed), fetch logs via SLS.
step3: For viewing the task instance list in the console: 1. If the pod is running, fetch the pod list via apiserver; 2. If the pod is deleted (completed), obtain the pod list from MySQL.

Why is this needed:
This approach should mitigate the issue of logs consuming excessive disk space on the node and solve the bug where logs are inaccessible post-SLS configuration.

I'm open to the possibility that my understanding of kubedl may not be fully comprehensive, so please correct me if I'm mistaken. Thanks for considering my feature request.

@llinvokerl llinvokerl changed the title [feature request] [feature request]integrating pod logs with SLS Nov 23, 2023
@llinvokerl llinvokerl changed the title [feature request]integrating pod logs with SLS [feature request] Integrating pod logs with SLS Nov 23, 2023
@SimonCqk
Copy link
Collaborator

@llinvokerl hi llinvokerl, would you contribute the proposed feature to the community? If so, I will assign it to you :)

@SimonCqk SimonCqk added the enhancement New feature or request label Nov 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants