Incorrect s3 metadata in Presigned URLs #1139
Comments
taylor1791
added
bug
|
Thanks for the issue. It looks like it is being added but not in the way you expect. A presigned request returned by the SDK also includes things like the method and HTTP headers to send. Dumping a debug version of the request shows the metadata is added as a header: println!("presigned req: {:#?}", presigned_request);Output: Whereas JS is hoisting all the headers it can into query parameters. If you convert the request directly into an HTTP request and send it with the headers it should work. Alternatively you can do something similar as JS is doing and hoist the returned headers and construct a new URI with them present as a workaround. I've opened #1140 to track potentially improving/adding this at some point. |
aajtodd
added
response-requested
|
Thanks for politely correcting my misunderstanding. Indeed it does work if I send the headers with the request. I've given the new issue a thumbs up. |
|
Comments on closed issues are hard for our team to see. |
Describe the bug
When I try to create a presigned URL for a put object operation with metadata, the metadata is not added to the object. Comparing the relevant parts of the generated presigned URL with another sdk implementation makes the reason evident.
Rust: X-Amz-SignedHeaders=host%3Bx-amz-meta-my-metadata
JS: X-Amz-SignedHeaders=host&x-amz-meta-my-metadata=my-value
Notice that the Rust URL appears to append the
x-amz-meta-my-metadataparam to theX-Amz-SignedHeaders=hostparam separated by an encoded semicolon? It also abstains from including the value.Entire Rust URL: https://my-bucket.s3.us-east-1.amazonaws.com/my-key?x-id=PutObject&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ANOTREAL%2F20240508%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240508T044525Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host%3Bx-amz-meta-my-metadata&X-Amz-Signature=ee22808d868692b2254b1ce4b66f5d5f075d1b179132c14e54b523cb7cacb250
Entire JS URL: https://my-bucket.s3.us-east-1.amazonaws.com/my-key?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=dummy%2F20240508%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240508T045407Z&X-Amz-Expires=3600&X-Amz-Signature=0241d68acc6847303e858bd8fd53ac306f726eda933bfe71c8a1efaaa920f146&X-Amz-SignedHeaders=host&x-amz-meta-my-metadata=my-value&x-id=PutObject
Expected Behavior
I expected the
presignedmethod to generate a URL that properly includes the specified metadata.Current Behavior
The metadata is improperly append to the X-Amz-SignedHeaders param.
Reproduction Steps
Possible Solution
No response
Additional Information/Context
No response
Version
Version
Environment details (OS name and version, etc.)
Linux 6.1.67
Logs
Logs
2024-05-08T05:08:20.701471Z DEBUG aws_sdk_s3::endpoint_lib: loading default partitions
2024-05-08T05:08:20.721860Z DEBUG hyper_rustls::config: with_native_roots processed 146 valid and 0 invalid certs
2024-05-08T05:08:20.722150Z DEBUG invoke{service=s3 operation=PutObject}:apply_configuration: aws_smithy_runtime::client::orchestrator: timeout settings for this operation: TimeoutConfig { connect_timeout: Set(3.1s), read_timeout: Disabled, operation_timeout: Disabled, operation_attempt_timeout: Disabled }
2024-05-08T05:08:20.722305Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime_api::client::interceptors::context: entering 'serialization' phase
2024-05-08T05:08:20.722442Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime_api::client::interceptors::context: entering 'before transmit' phase
2024-05-08T05:08:20.722515Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime::client::orchestrator: retry strategy has OKed initial request
2024-05-08T05:08:20.722549Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime::client::orchestrator: beginning attempt #1
2024-05-08T05:08:20.722626Z DEBUG invoke{service=s3 operation=PutObject}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator::endpoints: resolving endpoint endpoint_params=EndpointResolverParams(TypeErasedBox[!Clone]:Params { bucket: Some("my-bucket"), region: Some("us-east-1"), use_fips: false, use_dual_stack: false, endpoint: None, force_path_style: false, accelerate: false, use_global_endpoint: false, use_object_lambda_endpoint: None, key: Some("my-key"), prefix: None, disable_access_points: None, disable_multi_region_access_points: false, use_arn_region: None, use_s3_express_control_endpoint: None, disable_s3_express_session_auth: None }) endpoint_prefix=None
2024-05-08T05:08:20.722981Z DEBUG invoke{service=s3 operation=PutObject}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator::endpoints: will use endpoint Endpoint { url: "https://my-bucket.s3.us-east-1.amazonaws.com", headers: {}, properties: {"authSchemes": Array([Object({"name": String("sigv4"), "signingName": String("s3"), "signingRegion": String("us-east-1"), "disableDoubleEncoding": Bool(true)})])} }
2024-05-08T05:08:20.723173Z DEBUG invoke{service=s3 operation=PutObject}:try_op:try_attempt:lazy_load_identity: aws_smithy_runtime::client::identity::cache::lazy: identity cache miss occurred; added new identity (took 0ns) new_expiration=2024-05-08T05:23:20.721919Z valid_for=900s partition=IdentityCachePartition(0)
2024-05-08T05:08:20.723236Z DEBUG invoke{service=s3 operation=PutObject}:try_op:try_attempt: aws_smithy_runtime::client::identity::cache::lazy: loaded identity
2024-05-08T05:08:20.723678Z DEBUG invoke{service=s3 operation=PutObject}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator: ending orchestration early because the stop point is
BeforeTransmit2024-05-08T05:08:20.723778Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime::client::orchestrator: a retry is either unnecessary or not possible, exiting attempt loop
The text was updated successfully, but these errors were encountered: