Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace mime crate with mediatype crate #3239

Conversation

AdamSteinberg1
Copy link
Contributor

Motivation and Context

Fixes #2666
mime is not actively maintained, so I've replaced it with mediatype. mediatype also has the benefit that it is zero-copy and const constructible.

Description

I've replaced every instance of the Mime type with either MediaType or MediaTypeBuf. I've used MediaType when possible because it is zero-copy. I've also simplified logic in class ServerHttpBoundProtocolTraitImplGenerator to make all MediaTypes const constructed.

Testing

I changed the tests in rust-runtime/aws-smithy-http-server/src/protocol/mod.rs to use MediaType instead of Mime. I ran tests successfully with the following commands:

cargo test --manifest-path=rust-runtime/Cargo.toml
cargo clippy --manifest-path=rust-runtime/Cargo.toml
./gradlew codegen-core:check
./gradlew codegen-client:check
./gradlew codegen-server:check
./gradlew codegen-client-test:check
./gradlew codegen-server-test:check

Checklist

  • I have updated CHANGELOG.next.toml if I made changes to the smithy-rs codegen or runtime crates

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@AdamSteinberg1 AdamSteinberg1 requested review from a team as code owners November 19, 2023 15:25
@david-perez david-perez added the server Rust server SDK label Nov 20, 2023
Comment on lines 146 to 148
(t, s) if typ == t && subtype == s => true,
(t, "*") if typ == t => true,
("*", "*") => true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mediatype has constants too

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea originally my match statement looked like

            match (mim.ty, mim.subty) {
                (t, s) if typ == t && subtype == s => true,
                (t, mediatype::names::_STAR) if typ == t => true,
                (mediatype::names::_STAR, mediatype::names::_STAR) => true,
                _ => false,
            }

but this does not compile. You can't use mediatype constants in a match statement or else you get this error:

error: to use a constant of type `mediatype::Name<'_>` in a pattern, `mediatype::Name<'_>` must be annotated with `#[derive(PartialEq, Eq)]`
   --> aws-smithy-http-server/src/protocol/mod.rs:147:21
    |
147 |                 (t, mediatype::names::_STAR) if typ == t => true,
    |                     ^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: the traits must be derived, manual `impl`s are not sufficient
    = note: see https://doc.rust-lang.org/stable/std/marker/trait.StructuralEq.html for details

Our other option is to just use an if-else:

            if mim.ty == typ {
                mim.subty == subtype || mim.subty == mediatype::names::_STAR
            } else {
                mim.ty == mediatype::names::_STAR && mim.subty == mediatype::names::_STAR
            }

I prefer the way I wrote in the PR. I think it's more readable. But I'd be happy to change it to the if-else way, if that's what y'all want.

val (type, subtype) = contentType.split('/')
rustTemplate(
"""
const $staticContentType: #{MediaType}::MediaType = #{MediaType}::MediaType::new(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why isn't this a change in behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the same behavior, just simplified logic. Here's some code that is generated for the pokemon-serivce-server-sdk example.

CONTENT_TYPE_GETSTORAGE Before:

const CONTENT_TYPE_GETSTORAGE: ::mime::Mime = ::mime::APPLICATION_JSON;

CONTENT_TYPE_GETSTORAGE After:

const CONTENT_TYPE_GETSTORAGE: ::mediatype::MediaType = ::mediatype::MediaType::new(
    ::mediatype::Name::new_unchecked("application"),
    ::mediatype::Name::new_unchecked("json"),
);

CONTENT_TYPE_CAPTUREPOKEMON Before:

static CONTENT_TYPE_CAPTUREPOKEMON: ::once_cell::sync::Lazy<::mime::Mime> =
    ::once_cell::sync::Lazy::new(|| {
        "application/vnd.amazon.eventstream"
            .parse::<::mime::Mime>()
            .expect("BUG: MIME parsing failed, content_type is not valid")
    });

CONTENT_TYPE_CAPTUREPOKEMON After:

const CONTENT_TYPE_CAPTUREPOKEMON: ::mediatype::MediaType = ::mediatype::MediaType::new(
    ::mediatype::Name::new_unchecked("application"),
    ::mediatype::Name::new_unchecked("vnd.amazon.eventstream"),
);

For the mime crate, only certain predefined constants were const. For an arbitrary content type string, we had to use a static once_cell. For the mediatype crate all content types are const constructible, so we don't have to use once_cell anymore or handle special cases differently.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your point is fine in the case in which the content-type is valid. Right now the string you can see is exactly one of:

"application/json"
"application/octet-stream"
"application/vnd.amazon.eventstream"
"application/x-amz-json-1.0"
"application/x-amz-json-1.1"
"application/xml"
"image/jpeg"
"text/plain"

What happens if you find an invalid content-type, like type//x?
Currently your implementation is:

::mediatype::MediaType::new(
        ::mediatype::Name::new_unchecked("type"),
        ::mediatype::Name::new_unchecked(""),
    );

which constructs an invalid MediaType: MediaType { ty: Name("type"), subty: Name(""), suffix: None, params: [] }.

Your implementation, not to change behavior, should be:

::mediatype::MediaType::new(
        ::mediatype::Name::new("...").expect("..."),
        ::mediatype::Name::new("...").expect("..."),
    );

which parses the string and panics, as did the previous implementation when mime::parse failed.

all content types are const constructible, so we don't have to use once_cell anymore or handle special cases differently

If this is the case also for invalid strings, it's not worth it - or it has to be done in Kotlin.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea mediatype::Name::new() is not const while mediatype::Name::new_unchecked() is const. So there is no way to handle invalid strings using the const constructions from the mediatype crate. I think it's best to go back to the way it was previously written. That is, handle certain common cases with const constructors and handle the general case with a once_cell.

when (contentType) {
                    "application/json" -> "const $staticContentType: #{MediaType}::MediaType = #{MediaType}::media_type!(APPLICATION/JSON);"
                    "application/octet-stream" -> "const $staticContentType: #{MediaType}::MediaType = #{MediaType}::media_type!(APPLICATION/OCTET_STREAM);"
                    "application/x-www-form-urlencoded" -> 
                        """
                        const $staticContentType: #{MediaType}::MediaType = #{MediaType}::mediatype::new(
                            #{MediaType}::names::APPLICATION, 
                            #{MediaType}::Name::new_unchecked(\"x-www-form-urlencoded\")
                        );
                        """
                    else ->
                        """
                        static $staticContentType: #{OnceCell}::sync::Lazy<#{MediaType}::MediaType> = #{OnceCell}::sync::Lazy::new(|| {
                            #{MediaType}::MediaType::parse(${contentType.dq()}).expect("BUG: MIME parsing failed, content_type is not valid")
                        });
                        """
                }

@AdamSteinberg1 AdamSteinberg1 force-pushed the swap_mime_for_mediatype_crate branch from ade4131 to 48b4506 Compare December 5, 2023 02:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
server Rust server SDK
Projects
None yet
Development

Successfully merging this pull request may close these issues.

More efficient mime type parsing in server SDKs; avoid mime crate
3 participants