[开源之夏] adding image segmentation algorithms

[MiMO33]

Hi, my name is Tianyuan Qi, a CS student from China. I found the project "adding high-performance Image processing algorithms" in "summer of code". I have used the Matlab version "dct" & "idct" functions for creating digital image marks, OpenCV also have the function, but I haven't found the algorithms in Images.jl . So, I thought I could start with implementing these two algorithms.
What's your suggestion? Thanks.

[johnnychen94]

I never used them but I think they are already provided by FFTW.jl, see https://juliamath.github.io/FFTW.jl/latest/fft.html#FFTW.dct. Most basic image processing operations like this are scattered in the Julia ecosystem.

ImageMorphology, ImageSegmentation, ImageFeatures are some packages in JuliaImages that we don't have good support for. There can be many classic algorithms in these fields that we haven't implemented. Hence if you don't have an idea of what algorithms you should implement, maybe you can look around at those packages. Also, low-level image processing algorithms such as denoise, deblur and others are also missing in JuliaImages.

[MiMO33]

Sorry for the delay. There are so many image process algorithms, but finding the right ones to implement depends on the big picture. It's much more complicated than I first imagined. Thanks for your hint, it narrowed down my searching space.
I wanna focus on Image Segmentation.
Superpixels are becoming useful in many Computer Vision and Image processing algorithms. I haven't found any superpixel algorithms in Images library until I googled it and then found you implemented it,(Felzenswalb's Region Merging Algorithm doesn't count). Hence you already finished it, why don't you merge it into the library?
I'm still looking for the right job to do, don't worry, I didn't walk away. for now.

[johnnychen94]

Glad that you're still around!

I haven't found any superpixel algorithms in Images library until I googled it and then found you implemented it,(Felzenswalb's Region Merging Algorithm doesn't count). Hence you already finished it, why don't you merge it into the library?

Speaking of performance and generality, the types and structs I defined in johnnychen94/SuperPixels.jl#2 are probably not at their best version; it's written back to the days when I had a very limited understanding of Julia. I'll see if I get time to revisit it and perhaps register that package.

[johnnychen94]

Cause superpixel haven't merged into the library, could I do the work base on your code?

Of course, feel free to copy and paste the codes around (with appropriate comments) and start your own work on top of it. When needed, I can revisit your codes and make necessary changes.

[johnnychen94]

BTW, if you need any kind of help, feel free to ask questions via any form (QQ, mail, slack). To apply for the OSPP event, I encourage early demo codes to prove your ability of independent coding.

[johnnychen94]

smooth/denoise algorithms in https://github.com/sai-bi/ImageSmoothing are also good candidates.

[MiMO33]

After some thought, I think these four algorithms of Image Segmentation listed below might be okay:

1. Superpixel: with your previous work, I do think it's the best one to start.
2. Adaptthresh: Adaptive image threshold using local first-order statistics
3. Imsegkmeans: K-means clustering based image segmentation
4. Activecontour (GrabCut): Segment image into foreground and background using active contours (snakes) region growing technique (It's an extremely hard algorithm, I can feel it.)

As for the Image Smoothing Algorithms, thanks for your guide. It seems Julia Images library still lacks schedules of them. Let's talk about it after two or three Image Segmentation algorithms are finished.

I should work on the application on the official site and make the plan, the deadline is coming, time's tight for me, though the deadline is the first productive force.
To be honest, will the codes run well in the end? I don't know. But the design of juliaLang gave me lots of 'a-ha' moments, I respect the open-source movement and the rule of OSPP. I'll try my best.

After the algorithms were chosen, I'll find some papers, blogs, and the implements from other libraries, then write API and Pseudocode or demos. I'll write the application document parallelly. I assume the application document will give me a huge help.

Due to the unstable LaTeX performance with Chinese, I may write the proposal with Word in the first version. I hope you don't mind.
Thanks for your generous help and lead. Good Luck.

[johnnychen94]

All of them are good candidates to me, feel free to choose any of them. Depends on the difficulty of the algorithm, I'll say two or three algorithms will be enough so you can just focus on the segmentation project if that's your interest.

To be honest, will the codes run well in the end? I don't know.

OSPP is a training program so we don't have hard requirements on the output. The final target is to make the codes a part of JuliaImages, we'll (I guess mostly it is me at the moment) provide the necessary guidance to make sure this happens.

If you're new to JuliaImages, I encourage you to contribute some demos to https://juliaimages.org/dev/examples/. By doing that, we can provide some feedback on the code style, best practice, and others; it would be helpful before you put your hands on a big algorithm. Writing small demos is also a good way to let us know your coding ability and so that we know if you as the applicant can independently finish the OSPP project.

Due to the unstable LaTeX performance with Chinese, I may write the proposal with Word in the first version. I hope you don't mind.

I'm okay with this but better to submit PDF format. We're not doing academic publications so nobody will be nitpicking on this.

After the algorithms were chosen, I'll find some papers, blogs, and the implements from other libraries, then write API and Pseudocode or demos.

Sounds like a good plan to me.

I notice that the links you provided are from MathWorks. As a copyleft open source project (MIT license), we try to avoid the license issue, so be careful about the codes you're referencing and looking. More specifically, 1) Matlab source codes is not allowed 2) codes from other open source community can be used as long as you follow their license requirement 3) try not to use codes without a clear license, if must, ask for approval before using it.

[MiMO33]

If you're new to JuliaImages, I encourage you to contribute some demos to https://juliaimages.org/dev/examples/. By doing that, we can provide some feedback on the code style, best practice, and others; it would be helpful before you put your hands on a big algorithm. Writing small demos is also a good well to let us know your coding ability and so that we know if you as the applicant can independently finish the OSPP project.

Thanks for your useful advice, I think it's a good way to let jump into the development and fill the gap before writing for the lib directly. It's not that hard as implementing directly, demos could help other users directly. I'll work on it. I still fear my junky English expressing skills, it seems a challenge to me, but I may enjoy it.

As a copyleft open source project (MIT license), we try to avoid the license issue, so be careful about the codes you're referencing and looking. More specifically, 1) Matlab source codes is not allowed 2) codes from other open source community can be used as long as you follow their license requirement 3) try not to use codes without a clear license, if must, ask for approval before using it.

The define of copying code is a little fuzzy to me, I know we can't copy the original codes, but what about the idea behind the code? That's a disputed zone.

[MiMO33]

If you're new to JuliaImages, I encourage you to contribute some demos to https://juliaimages.org/dev/examples/. By doing that, we can provide some feedback on the code style, best practice, and others; it would be helpful before you put your hands on a big algorithm. Writing small demos is also a good well to let us know your coding ability and so that we know if you as the applicant can independently finish the OSPP project.

I have writen a draft demo of juliaimage, you can find it here . The purpose of writing a demo is evaluating about the OSPP project, so I didn't pull it to the main repo.

By writing the demo, I do learned more about the ecosystem. Your GSoC2019 repo explains many things, but I still can't write good demos due to my poor understand of the project and julia. I need some time to figure out the project.

[johnnychen94]

Although OSPP is a training program like GSoC, we still require students to independently code the proposed algorithms. That's why we want to see a demo from all potential applicants. The demo you gave is just a copy of existing documentation https://juliaimages.org/dev/pkgs/segmentation/ so that doesn't prove anything.

Only one student can be selected for each OSPP project, so definitely there will be competition. There already is, see #964. The demo I want to see from you is something like this to show that you understand the algorithms and you can independently put pieces together to implement something that doesn't yet exist in Julia.

I won't make a decision until I see the proposals, i.e., until June 14. There are still two weeks of time for you to learn and prepare. You're welcomed to learn and use JuliaImages and I'll be around and I'll be very happy to offer help (I really want to see more Chinese users use Julia). If you refrain from the competition, I'll understand, I don't like it either.

Only one student can be selected for each OSPP project

We didn't know this rule until the mentor project submission period ended, which is exactly why there's only one such project. Otherwise, I'll submit multiple ones so as to avoid unnecessary competition.