Project Link: AI Photo Album Website
This is a photo album web application, that can be searched using natural language through both text and voice. Everytime users upload a photo, features and labels inside the photo will be detected and stored. When the user later search these labels in this album, relevant photos with corresponding features will come up.
This project mainly used services provided by AWS.
The function of photo searching via natural language is implemented with Amazon Lex.
Here're the steps:
- Create a Lex bot in AWS console (Using Lex V2 is highly recommended)
- Create an intent in the bot
- Add training utterances to the intent, such that the bot can pick up both keyword searches (“trees”, “birds”), as well as sentence searches (“show me trees”, “show me photos with trees and birds in them”). Use two slots 'FirstPhotoIntent' and 'SecondPhotoIntent' to capture the keywords entered by users.
- Check the option 'Required for this intent' for the first slot only.
Lex Utterances
Slots
- Check the image restriction of Rekognition. Make sure your images is supported by Rekognition.
- Choose the photo you want to upload and add custom labels if you want. Click 'upload' to upload.
- After the photo is uploaded, Rekognition will analyze it and generate labels for it. The labels of photos are stored in the OpenSearch index.
- Query photos according to utterances in Lex bot. Results will apear at the bottom.
Video Demo: https://youtu.be/Xep-HgO3IYk
Album Interface
Labels of Photos
Query Example
In this step, I choose GitHub as the source. The purpose is that every time I push a new commit to my GitHub repository, it will trigger the pipeline to build and deploy my new commit.
After choosing Github (V2) as the source, the settings will guide developers to connect with their GitHub repository. The settings is shown as below.
Select CodeBuild as the Build provider. If you haven't created any build project for this pipeline, click 'Create project'. For the build project creation, I choose Managed Image, EC2, Ubuntu, New Service Role and 'Use a buildspec file'. Remember to add permission of accessing lambda service to the new service role.
Moreoevr, the correctness of buildspec.yml file is important to the success of the build stage.
In my buildcpec.yml, I separately packaged the dependencies and code for lambda function 1 (line 18) and lambda function 2 (line 19) and write the command of updating my lambda functions in AWS console (line 24, 25).
After the creation finished, we return to the page of step 3 and choose the poject name that we just created.
In this step, I choose S3 as the deploy provider. Before continuing, create a S3 bucket and select this bucket in this stage. The S3 bucket will be used to store files from your GitHub repository.
After the above steps are finished, you should be able to see the stage of source, build and deploy succeed after you push a commit to the repository selected in step 2.
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Letian Jiang (Email: [email protected])
Shihui Huang (Email: [email protected])
The following recourses helped me to complete this project.