Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor EncryptedDiskStore to return io.Reader instead of io.ReadCloser #4329

Closed

Conversation

AndersonQ
Copy link
Member

What does this PR do?

Refactors EncryptedDiskStore to return io.Reader instead of io.ReadCloser

Why is it important?

Checklist

  • My code follows the style guidelines of this project
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • ~~[ ] I have made corresponding change to the default configuration files
  • [ ] I have added tests that prove my fix is effective or that my feature works~~
  • [ ] I have added an entry in ./changelog/fragments using the changelog tool
  • [ ] I have added an integration test or an E2E test

Related issues

Questions to ask yourself

  • How are we going to support this in production?
  • How are we going to measure its adoption?
  • How are we going to debug this?
  • What are the metrics I should take care of?
  • ...

On Windows if Load() is called, the returned reader isn't closed and then Save() is called, Save() would fail.
By returning just a io.Reader this possible failure scenario is prevented.
@AndersonQ AndersonQ self-assigned this Feb 23, 2024
@AndersonQ AndersonQ requested a review from a team as a code owner February 23, 2024 10:25
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

@AndersonQ AndersonQ requested review from pchila and removed request for blakerouse February 23, 2024 10:26
@@ -97,7 +97,7 @@ func (d *DiskStore) Save(in io.Reader) error {
}

// Load return a io.ReadCloser for the target file.
func (d *DiskStore) Load() (io.ReadCloser, error) {
func (d *DiskStore) Load() (io.Reader, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you sure this is allright?

Copy link
Member

@pchila pchila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see that using os.ReadFile instead of opening the file with os.O_RDONLY removes the need for Close() on the io.Reader we return from Load(), however looking at the code it seems that in most places the Reader (or ReadCloser) is passed to config.NewConfigFrom() which calls io.ReadAll() before unmarshalling.

With this modification we are doubling the buffers with the diskstore content (one in the new ByteReader + the one in config.NewConfigFrom, I am worried that we are churning more memory as the EncryptedDiskStores are created in many places throughout the code.

Maybe we should have a small benchmark of EncryptedDiskStore.Load() + config.NewConfigFrom to have a better idea of the memory impact

@AndersonQ AndersonQ closed this Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants