Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Are there requirements for the input image size?(question) #193

Open
jasan-s opened this issue Jun 11, 2019 · 8 comments
Open

Are there requirements for the input image size?(question) #193

jasan-s opened this issue Jun 11, 2019 · 8 comments
Labels
question Further information is requested

Comments

@jasan-s
Copy link

jasan-s commented Jun 11, 2019

For the mobile net feature extractor and for using the knn classifier are there any input image requirements? i.e can provide any aspect ratio and resolution image to the mobile net and the knn classifier? Or is it required that a 1:1 ratio image is served but no requirement for the resolution size. Or is there a limit to the max resolution as well.

@joeyklee
Copy link
Contributor

@jasan-s - Great question! Something we should also point to in our documentation.

In general, here's what I would say (@yining1023 feel free to correct me if I'm wrong!):

  • resolution @ 72 dpi I think is good/fine. Higher resolution probably won't help you more except make the processing times longer.
  • Image size size wise, probably better to not have gigantic images (> 640px x 480px), so smaller the images the more manageable (< 640px x 480px) .

I hope this helps! I will keep this open so that we make sure to add:

  1. A note about image resolution and size
  2. A note about the number of training data limits (seems also to be a common question that pops up), see: KNN classifier save seems capped at around 4000 examples ml5-library#405, and Adding Dog dataset upto 15830 images Knn Classifier Feature Extractor ml5-library#356

Thank you!

@yining1023
Copy link
Member

Hi @jasan-s, Good question!
I agree with @joeyklee. There is no specific requirement for image size or resolution.

From the library source code point of view:

  • For the mobile net feature extractor: ml5.featureExtractor(‘mobileNet’).addImage(img) will crop the image from the center in 224x224 px.
  • For KNN classifier, it doesn't transform the image in any way.

@jasan-s
Copy link
Author

jasan-s commented Jun 11, 2019

@yining1023 @joeyklee agree that it would be great addition to the docs. Since mobile net crops a square 224 px from the center it would be best to enforce the input(i.e live webcam feed) to be a square ratio. Because if the important area of the feature for classification lies outside the square then user adding those examples to model would be meaningless.
Would you both agree with the following steps of implementation:

  1. Force users webcam feed to be 1:1 square ratio (i.e 512x512) higher res for better user experience
  2. user clicks a button to train a class for KNN classifier. The 512x512 image is scaled to 214x214 sent to mobile net feature extractor. And the features are added to knn model. (User still seeing the higher res feed)
    3)For testing predictions the user still sees the 512x512 feed and the same scaling down happens.

Important difference is that what user sees on screen is higher resolution but the same 1:1 square ratio then what is being used for building the model.

@shiffman
Copy link
Member

Because we want ml5 to be easy for beginners, we made the decision to handle all of the image cropping and resizing behind the scenes when sending an image to MobileNet. However, we probably should be more clear in the documentation as to what is going on as well as consider consistency with other systems. This came up when working with @irealva on compatibility with the new teachable machine which by default flips the webcam's image!

(@yining1023 re: KNN, I believe anytime an image is sent to MobileNet it has to be a 224x224 square image? KNNclassifier can, of course, accept other forms of data, but in terms of the features of an image extracted from MobileNet, the same constraints apply, yes?)

@yining1023
Copy link
Member

@shiffman yes! That is correct.

@jasan-s
Copy link
Author

jasan-s commented Jun 11, 2019

@shiffman agree that handling image normalizing within lib is good decision. seems to be done here mobilenetInfer function using image.resize_bilinear.

@shiffman @yining1023 @joeyklee Can you please confirm if the the below statements are true:

Statement: No mater what size and aspect ratio image is inputed to ml5, there is NEVER any image data loss in normalizing and resizing because align_corners is set true?

Result: If above is true then ml5 users don't have to do any image manipulation and can carelessly send any image.

@bomanimc bomanimc added the question Further information is requested label Nov 25, 2020
@sproutleaf sproutleaf transferred this issue from ml5js/ml5-library Jul 11, 2023
@sproutleaf
Copy link

Transferred issue from ml5-library to ml5-website because the question has been answered and now we need to update the documentation.

@devihall
Copy link

devihall commented Apr 4, 2024

@sproutleaf > Transferred issue from ml5-library to ml5-website because the question has been answered and now we need to update the documentation.

Not seeing the answers/response to questions from @jasan-s in #193 (comment)

"Can you please confirm if the the below statements are true:

Statement: No mater what size and aspect ratio image is inputed to ml5, there is NEVER any image data loss in normalizing and resizing because align_corners is set true?

Result: If above is true then ml5 users don't have to do any image manipulation and can carelessly send any image."

Am I missing something? I'm also interested in figuring out the optimal size and resolution for our project

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

7 participants