-
Notifications
You must be signed in to change notification settings - Fork 112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid communicating with S3 unless necessary #616
Avoid communicating with S3 unless necessary #616
Conversation
@glenrobson this works and tested over current develop branch. But I would love to have your larger pull merged first and test with it and ideally give this a spin on a production server with real load to see if having a get Object Info from S3 fail last (when deferred) has any other consequences, if that is OK |
@garyttierney hi. I rebased against |
Previously the S3 source would eagerly connect to S3 retrieve information about the object being requested when creating a stream. This change defers retrieval of properties from S3 until needed by the S3StreamFactory, and allows cached files to be served directly from disk with no intermediate S3 lookups.
28f96d2
to
397de41
Compare
Yep no problem, rebased against latest changes in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Works and checks (finally!) are all passing
@garyttierney thanks for your contribution! |
Previously the S3 source would eagerly connect to S3 retrieve information about the object being requested when creating a stream. This change defers retrieval of properties from S3 until needed by the S3StreamFactory, and allows cached files to be served directly from disk with no intermediate S3 lookups.
It's a small optimization, but can shave a dozen or so MS off of a cached image request.