Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed issue of re-reading rotated files #302

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

punxism
Copy link

@punxism punxism commented Sep 17, 2021

This is a fix for #238

Premis

  1. logstash checks the file uniqueness using the inode number, so even if sincedb exists, it does not fully support changing the file name in all cases.
  2. In a cloud environment, app rebooting occurs frequently due to deployment using docker, and as a result, tens of gigabytes or peta logs are read again.

Solution

  1. If the path of sincedb is changed and the inode is the same, it is recognized as a different file and the file is read again.
  2. However, even when file rotation occurs, it is always read again, which caused a problem. To solve the problem, look at the path of sincedb and start with this path and if the inode is not changed, it is regarded as a rotated file and the value of sincedb is maintained.

@cla-checker-service
Copy link

cla-checker-service bot commented Sep 17, 2021

❌ Author of the following commits did not sign a Contributor Agreement:
a439c21

Please, read and sign the above mentioned agreement if you want to contribute to this project

@punxism
Copy link
Author

punxism commented Sep 23, 2021

❌ Author of the following commits did not sign a Contributor Agreement:
5040fab

Please, read and sign the above mentioned agreement if you want to contribute to this project

Done.

@punxism punxism force-pushed the fix-rorated-log-rereading branch from 5040fab to a439c21 Compare September 23, 2021 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants