-
Notifications
You must be signed in to change notification settings - Fork 5
Design Documentation
Version | Date | Description |
---|---|---|
Rev 1.0 | 2023-10-08 | Initial version |
Rev 1.1 | 2023-10-19 | Updated implemented Auth & Post APIs details |
Rev 1.2 | 2023-11-02 | Updated high-level system architecture |
Rev 1.3 | 2023-11-05 | Updated implemented Follow & Search APIs details |
Rev 1.4 | 2023-11-05 | Updated Android class diagram |
Rev 1.4.1 | 2023-11-05 | Updated Android implementation details |
Rev 1.5 | 2023-11-19 | Updated Follow APIs details and implemented notification API |
Rev 1.5.1 | 2023-11-19 | Updated Password APIs |
Rev 1.6 | 2023-11-29 | Updated Backend Testing result |
Rev 1.6.1 | 2023-11-29 | Updated Frontend Testing |
Rev 1.7 | 2023-12-03 | Updated Frontend Class Diagram |
Rev 1.8 | 2023-12-10 | Added UAT document |
Rev 1.9 | 2023-12-10 | Updated Notification deletion APIs |
Rev 1.10 | 2023-12-10 | Updated System Architecture and Diagrams |
Rev 1.11 | 2024-03-14 | Updated AI Architecture and Diagrams |
Our project consists of a front-end and a back-end, and some functions depend on external services (Stable Diffusion, Notifee, etc.). Page views are implemented in Kotlin with Retrofit for HTTP communication with the API server. The backend server is deployed on AWS EC2 instances mainly using FastAPI and PostgreSQL (Docker image). AWS S3 is used for storing images. Inference server is used for generating images using Stable Diffusion, which is served by NVIDIA Triton inference server and Kubernetes.
The above diagram is the MVC diagram. Our project follows MVC design pattern and we will request or respond JSON data from Android views to server models via API endpoints.
The following class diagram depicts relationships and dependencies of activities, fragments and data classes of Android project. One thing to note is that activities and fragments that are covered with dotted lines imply that there exists an architecture inside each of them, including UI layer and data layer. UserDetail class is a child class of User, which includes more detailed informations of a specific user, whereas User includes only username and small profile image. This separation efficiently relieves the data size when retrieving post or comment data.
Overall backend structure is consisted of router, service, repository, schema, models, utils and etc.. The router directs HTTP requests to appropriate handlers, defining endpoints and methods. The service layer contains the business logic of each APIs and repository helps service layer to interact with databases, handling data storage and retrieval. Schema defines the valid request and response format. Model defines the structure of our object-relational database. Lastly, utils include various helper functions for tasks like AWS connection and token generation. Each component plays a distinct role, contributing to a well-organized, maintainable backend system.
The E-R diagram depicts the database model comprising various entities such as users, posts, comments, likes, follows, blocks, notifications, and images. These entities have associated attributes representing user details, post content, like and comment counts, timestamps, and more. Relationships exist between these entities, defining connections like a post belonging to a user, a comment related to a post, a like associated with a post or comment, and follow/block actions involving users.
Below is the description for each model.
Model | Description |
---|---|
User | The user. Has bio and profile image. Can create a post by creating a new image from an existing image by writing a prompt. Can have followers and followings. Can like posts and comments. |
Post | Posts with newly created image as prompted and content by users |
Comment | Comments on posts by users |
Image | Contains original image, masked image, newly created image, and used prompt |
Notification | Notifications for follows or likes |
PostLike | Likes on posts by users |
CommentLike | Likes on comments by users |
Follow | A user following another user |
Class | Method | Parameters | return | Description |
---|---|---|---|---|
LoginActivity | login() | email: String, password: String | Login with given email and password | |
VerifyEmailActivity | verifyCode() | code: String | Verify email with 6 digit code | |
VerifyEmailActivity | startTimer() | Starts timer to send email again | ||
SignUpActivity | register() | email: String, password: String, username: String | Register with the given email, password, and username | |
FeedFragment | fetchPosts() | List<Post> | List the post in the feed | |
FeedFragment | like() | post: Post | Like or unlike the post | |
PostFragment | comment() | post: Post, content: String | Leaves a comment on current post | |
PostFragment | editPost() | postId: String, content: String | Edit post content | |
PostFragment | deletePost() | postId: String | Delete post | |
PostFragment | deleteComment() | commentId: String | Delete comment | |
ProfileFragment | fetchUserData() | user: UserProfileDetail, userStatus: String | Show user profile with three statuses: mine, followee's, other user's | |
ProfileFragment | sendFollowRequest() | userId: String | Send a follow request to other user | |
ProfileFragment | cancelFollowRequest() | userId: String | Cancel follow request | |
ProfileFragment | unfollow() | user: UserProfileDetail | Unfollow a user | |
EditProfileActivity | editProfile() | profileImage: Base64, bio: String, username: String | Edit profile information | |
EditProfileActivity | logout() | Log out from current user | ||
EditProfileActivity | unregister() | Log out and delete account | ||
SearchFragment | searchUser() | searchQuery: String | Search user with a given query | |
NotificationFragment | fetchNotifications() | List<Notification> | Show notifications list | |
NotificationFragment | acceptFollowRequest() | userId: String | Accept follow request | |
NotificationFragment | declineFollowRequest() | userId: String | Decline follow request | |
MainActivity | selectImage() | Select an image from gallery | ||
MainActivity | generateImage() | prompt: String | Generate image (Text to image) | |
CameraActivity | onTakePhoto() | Takes photo | ||
CameraActivity | onSwitchCamera() | Switches current camera between rear and front | ||
ImageEditActivity | drawMask() | Draw a mask on an image | ||
ImageEditActivity | inference() | originalImage: Base64, mask: Base64, prompt: String | Run AI model to inference and create a modified image | |
ImageEditActivity | fetchPromptSuggestion() | originalImage | Suggest prompts from an image | |
WriteTextActivity | uploadPost() | editedImage, content: String | Upload a post with a modified image | |
WriteTextActivity | regenerate() | Regenerate images with same information | ||
WriteTextActivity | download() | Download image to gallery | ||
Post | comment() | id: int, content: String, user: UserProfile | Write a comment | |
Post | like() | id: int, user: UserProfile | Like a post | |
Post | update() | id: int, content: String, imageUrl: String | Update the post with given content and image | |
Post | delete() | id: int | Delete the post | |
Comment | like() | id: int, user: UserProfile | Like a comment | |
Comment | update() | id: int, content: String | Update the comment with given content | |
Comment | delete() | id: int | Delete the comment |
Name | Domain | Method | URL |
---|---|---|---|
Register | Auth | POST | /auth/register |
Login | Auth | POST | /auth/login |
Check Email and Username exits | Auth | GET | /auth/check |
Token Verification | Auth | POST | /auth/verify |
Token Refresh | Auth | POST | /auth/refresh |
Logout | Auth | GET | /auth/logout |
Unregister | Auth | POST | /auth/unregister |
Send Code Email | Auth | POST | /auth/email |
Code Verification | Auth | POST | /auth/verify/code |
Password Verification | Auth | POST | /auth/verify/password |
Change Password | Auth | PATCH | /auth/password |
Get My Profile | User | GET | /user/me |
Get User Profile | User | GET | /user/{user_id} |
Edit My Profile | User | PATCH | /user/me |
Request Follow | User | POST | /user/{user_id}/follow_request |
Cancel Follow Request | User | DELETE | /user/{user_id}/cancel_request |
Accept Follow Request | User | POST | /user/{user_id}/accept_request |
Reject Follow Request | User | POST | /user/{user_id}/reject_request |
Unfollow User | User | DELETE | /user/{user_id}/unfollow |
Reject Follow | User | DELETE | /user/{user_id}/reject_follow |
Get User Follow Info | User | GET | /user/{user_id}/follow_info |
Get Followers | User | GET | /user/{user_id}/followers |
Get Followings | User | GET | /user/{user_id}/followings |
Get Posts with pagination | Post | GET | /post |
Get My Posts with pagination | Post | GET | /post/me |
Get Other User Posts with pagination | Post | GET | /post/{user_id} |
Create Post | Post | POST | /post |
Get Post | Post | GET | /post/{post_id} |
Update Post Content | Post | PATCH | /post/{post_id} |
Delete Post | Post | DELETE | /post/{post_id} |
Get Comments with pagination | Post | GET | /post/{post_id}/comment |
Create Post Comment | Post | POST | /post/{post_id}/comment |
Update Post Comment | Post | PATCH | /post/comment/{comment_id} |
Delete Post Comment | Post | DELETE | /post/comment/{comment_id} |
Like Post | Post | POST | /post/{post_id}/like |
Like Comment | Post | POST | /post/comment/{comment_id}/like |
Search User | Search | GET | /search/user/{search_string} |
Get My Notifications | Notification | GET | /notification/me |
Delete My All Notification | Notification | DELETE | /notification/me |
Delete My Specific Notification | Notification | DELETE | /notification/{notification_id} |
1. Auth
1-1. Register
-
Endpoint:
/auth/register
-
Request (
POST
){ "email": "string", "password": "string", "username": "string" }
-
Response
-
Status:
200 OK
{ "access_token": "string", "refresh_token": "string", "user_id": "string", "username": "string" }
-
Status:
400 Bad Request
{ "error_code": 400, "message": "USER__DUPLICATE_EMAIL_OR_USERNAME", }
-
1-2. Login
-
Endpoint:
/auth/login
-
Request (
POST
){ "email": "string", "password": "string" }
-
Response
-
Status:
200 OK
{ "access_token": "string", "refresh_token": "string", "user_id": "string", "username": "string" }
-
Status:
401 Unauthorized
{ "error_code": 401, "message": "USER__PASSWORD_DOES_NOT_MATCH", }
-
Status:
404 Not Found
{ "error_code": 404, "message": "USER__NOT_FOUND", }
-
1-3. Check Email and Username exits
-
Endpoint:
/auth/check
-
Request (
GET
){ "email": "string" "username": "string" }
-
Response
-
Status:
200 OK
{ "email_exits": true "username_exits": false }
-
1-4. Token Verification
-
Endpoint:
/auth/verify
-
Request (
POST
){ "access_token": "string" }
-
Response
-
Status:
200 OK
{}
-
Status:
400 Bad Request
{ "error_code": 400, "message": "TOKEN__DECODE_ERROR", }
-
Status:
400 Bad Request
{ "error_code": 400, "message": "TOKEN__EXPIRED_TOKEN", }
-
1-5. Token Refresh
-
Endpoint:
/auth/refresh
-
Request (
POST
){ "access_token": "string" "refresh_token": "string" }
-
Response
-
Status:
200 OK
{ "access_token": "string" "refresh_token": "string" }
-
Status:
400 Bad Request
{ "error_code": 400, "message": "TOKEN__DECODE_ERROR", }
-
1-6. Logout
-
Endpoint:
/auth/logout
-
Request (
GET
){}
-
Response
-
Status:
200 OK
{ "message" : "Logout Success" }
-
Status:
401 Unauthorized
{ "error_code": 401, "message": "UNAUTHORIZED", }
-
1-7. Unregister
-
Endpoint:
/auth/unregister
-
Request (
DELETE
){}
-
Response
-
Status:
200 OK
{ "message" : "User {req.user.id} deleted successfully" }
-
Status:
401 Unauthorized
{ "error_code": 401, "message": "UNAUTHORIZED", }
-
Status:
404 Not Found
{ "error_code": 404, "message": "USER__NOT_FOUND", }
-
1-8. Send Code Email
-
Endpoint:
/auth/email
-
Request (
POST
){ "email": [ "[email protected]" ] }
-
Response
-
Status:
200 OK
{ "message" : "Email sent to {email}" }
-
Status:
400 Bad Request
{ "error_code": 400, "message": "USER__DUPLICATE_EMAIL_OR_USERNAME", }
-
1-9. Code Verification
-
Endpoint:
/auth/verify/code
-
Request (
POST
){ "email": "string", "code": 012345, }
-
Response
-
Status:
200 OK
{ "message" : "Code verified successfully" }
-
Status:
404 Not Found
{ "error_code": 404, "message": "CODE__NOT_FOUND", }
-
Status:
400 Bad Request
{ "error_code": 400, "message": "CODE__EXPIRED", }
-
1-10. Password Verification
-
Endpoint:
/auth/verify/password?password={password}
-
Request (
POST
) -
Response
-
Status:
200 OK
{ "message" : "Password verified successfully" }
-
Status:
401 Unauthorized
{ "error_code": 401, "message": "USER__PASSWORD_DOES_NOT_MATCH", }
-
1-11. Change Password
-
Endpoint:
/auth/verify/password?new_password={new_password}
-
Request (
PATCH
) -
Response
-
Status:
200 OK
{ "message" : "Password changed successfully" }
-
2. User
2-1. Get My Profile
-
Endpoint:
/user/me
-
Request (
GET
){}
-
Response
-
Status:
200 OK
{ "id": "string", "email" : "string", "username": "string", "bio": "string", "profile_image_url": "string", }
-
2-2. Get User Profile
-
Endpoint:
/user/{user_id}
-
Request (
GET
){}
-
Response
-
Status:
200 OK
{ "id": "string", "email" : "string", "username": "string", "bio": "string", "profile_image_url": "string", }
-
2-3. Edit User Profile
-
Endpoint:
/user/me
-
Request (
PATCH
){ "user_update" : { "username": "string", "bio": "string", "profile_image_url": "string", }, "file" : <UploadFile> }
-
Response
-
Status:
200 OK
{ "id": "string", "email" : "string", "username": "string", "bio": "string", "profile_image_url": "string", }
-
2-4. Request Follow
-
Endpoint:
/user/{user_id}/follow_request
-
Request (
POST
){}
-
Response
- Status:
200 OK
{ "message" : "Requested user {user_id} follow" }
- Status:
404 Bad Request
{ "error_code": 404, "message": "USER__NOT_FOUND", }
{ "error_code": 404, "message": "FOLLOW__MYSELF", }
{ "error_code": 404, "message": "FOLLOW__ALREADY_EXISTS", }
- Status:
2-5. Cancel Follow Request
-
Endpoint:
/user/{user_id}/cancel_request
-
Request (
DELETE
){}
-
Response
- Status:
200 OK
{ "message" : "Canceled user {user_id} follow request" }
- Status:
404 Bad Request
{ "error_code": 404, "message": "FOLLOW__NOT_FOUND", }
{ "error_code": 404, "message": "FOLLOW__WRONG_STATUS", }
- Status:
2-6. Accept Follow Request
-
Endpoint:
/user/{user_id}/cancel_request
-
Request (
POST
){}
-
Response
- Status:
200 OK
{ "message" : "Accepted user {user_id} follow request" }
- Status:
404 Bad Request
{ "error_code": 404, "message": "FOLLOW__NOT_FOUND", }
{ "error_code": 404, "message": "FOLLOW__ALREADY_ACCEPTED", }
- Status:
2-7. Reject Follow Request
-
Endpoint:
/user/{user_id}/reject_request
-
Request (
DELETE
){}
-
Response
- Status:
200 OK
{ "message" : "Rejected user {user_id} follow request" }
- Status:
404 Bad Request
{ "error_code": 404, "message": "FOLLOW__NOT_FOUND", }
{ "error_code": 404, "message": "FOLLOW__WRONG_STATUS", }
- Status:
2-8. Unfollow User
-
Endpoint:
/user/{user_id}/unfollow
-
Request (
DELETE
){}
-
Response
- Status:
200 OK
{ "message" : "Unfollowed user {user_id}" }
- Status:
404 Bad Request
{ "error_code": 404, "message": "FOLLOW__NOT_FOUND", }
{ "error_code": 404, "message": "FOLLOW__WRONG_STATUS", }
- Status:
2-9. Reject Follow
-
Endpoint:
/user/{user_id}/reject_follow
-
Request (
DELETE
){}
-
Response
- Status:
200 OK
{ "message" : "Rejected user {user_id} follow" }
- Status:
404 Bad Request
{ "error_code": 404, "message": "FOLLOW__NOT_FOUND", }
{ "error_code": 404, "message": "FOLLOW__WRONG_STATUS", }
- Status:
2-10. Get User Follow Info
-
Endpoint:
/user/{user_id}/follow_info
-
Request (
GET
){}
-
Response
- Status:
200 OK
{ "follower_cnt": int, "following_cnt": int, "follower_status": int, "following_status": int }
- Status:
2-11. Get Followers
-
Endpoint:
/user/{user_id}/followers
-
Request (
GET
){}
-
Response
- Status:
200 OK
{ "total": 0, "items": [ { "email": "string", "id": "string", "username": "string", "profile_image_url": "string", "is_follower": bool, "is_following": bool } ], "next_cursor": 0 }
- Status:
2-12. Get Followings
-
Endpoint:
/user/{user_id}/followings
-
Request (
GET
){}
-
Response
- Status:
200 OK
{ "total": 0, "items": [ { "email": "string", "id": "string", "username": "string", "profile_image_url": "string" "is_follower": bool, "is_following": bool } ], "next_cursor": 0 }
- Status:
3. Post
3-1. Get Posts with pagination
-
Endpoint:
/post
-
Request (
GET
){}
-
Status:
200 OK
{ "total": 0, "items": [ { "content": "string", "image_url": "string", "origin_image_url": "string", "mask_image_url": "string", "id": "string", "created_at": "2023-11-30T07:03:10.040Z", "updated_at": "2023-11-30T07:03:10.040Z", "user": { "id": "string", "username": "string", "profile_image_url": "string" }, "like_cnt": 0, "comment_cnt": 0, "is_liked": -1 } ], "next_cursor": 0 }
3-2. Get My Posts with pagination
-
Endpoint:
/post/me
-
Request (
GET
){}
-
Status:
200 OK
{ "total": 0, "items": [ { "content": "string", "image_url": "string", "origin_image_url": "string", "mask_image_url": "string", "id": "string", "created_at": "2023-11-30T07:03:10.040Z", "updated_at": "2023-11-30T07:03:10.040Z", "user": { "id": "string", "username": "string", "profile_image_url": "string" }, "like_cnt": 0, "comment_cnt": 0, "is_liked": -1 } ], "next_cursor": 0 }
3-3. Get Other User Posts with pagination
-
Endpoint:
/post/{user_id}
-
Request (
GET
){}
-
Status:
200 OK
{ "total": 0, "items": [ { "content": "string", "image_url": "string", "origin_image_url": "string", "mask_image_url": "string", "id": "string", "created_at": "2023-11-30T07:03:10.040Z", "updated_at": "2023-11-30T07:03:10.040Z", "user": { "id": "string", "username": "string", "profile_image_url": "string" }, "like_cnt": 0, "comment_cnt": 0, "is_liked": -1 } ], "next_cursor": 0 }
3-4. Create Post
-
Endpoint:
/post
-
Request (
POST
){ "post": { "content": "string" }, "image": { "modified_image": "string", "origin_image": "string", "mask_image": "string", "prompt": "string" } }
-
Response
- Status:
200 OK
{ "content": "string", "image_url": "string", "origin_image_url": "string", "mask_image_url": "string", "id": "string", "created_at": "2023-11-30T07:12:04.091Z", "updated_at": "2023-11-30T07:12:04.091Z", "user": { "id": "string", "username": "string", "profile_image_url": "string" }, "like_cnt": 0, "comment_cnt": 0, "is_liked": -1 }
- Status:
3-5. Get Post
-
Endpoint:
/post/{post_id}
-
Request (
GET
){}
-
Response
- Status:
200 OK
{ "content": "string", "image_url": "string", "origin_image_url": "string", "mask_image_url": "string", "id": "string", "created_at": "2023-11-30T07:12:04.091Z", "updated_at": "2023-11-30T07:12:04.091Z", "user": { "id": "string", "username": "string", "profile_image_url": "string" }, "like_cnt": 0, "comment_cnt": 0, "is_liked": -1 }
- Status:
3-6. Update Post Content
-
Endpoint:
/post/{post_id}
-
Request (
PATCH
){ "content": "string", "image_url": "string", "origin_image_url": "string", "mask_image_url": "string" }
-
Response
- Status:
200 OK
{ "content": "string", "image_url": "string", "origin_image_url": "string", "mask_image_url": "string", "id": "string", "created_at": "2023-11-30T07:12:04.091Z", "updated_at": "2023-11-30T07:12:04.091Z", "user": { "id": "string", "username": "string", "profile_image_url": "string" }, "like_cnt": 0, "comment_cnt": 0, "is_liked": -1 }
- Status:
3-7. Delete Post
-
Endpoint:
/post/{post_id}
-
Request (
DELETE
){}
-
Response
- Status:
200 OK
{ "message": "Post {post_id} deleted successfully" }
- Status:
404 Not Found
{ "error_code": 404, "message": "POST__NOT_FOUND" }
- Status:
3-8. Get Comments with pagination
-
Endpoint:
/post/{post_id}/comment
-
Request (
GET
){}
-
Response
-
Status:
200 OK
{ "total": 0, "items": [ { "post_id": "string", "content": "string", "id": "string", "created_at": "2023-11-30T07:19:50.995Z", "updated_at": "2023-11-30T07:19:50.995Z", "user": { "id": "string", "username": "string", "profile_image_url": "string" }, "like_cnt": 0, "is_liked": -1 } ], "next_cursor": 0 }
-
3-9. Create Post Comment
-
Endpoint:
/post/{post_id}/comment
-
Request (
POST
){ "content": "string" }
-
Response
-
Status:
200 OK
{ "post_id": "string", "content": "string", "id": "string", "created_at": "2023-11-30T07:19:50.995Z", "updated_at": "2023-11-30T07:19:50.995Z", "user": { "id": "string", "username": "string", "profile_image_url": "string" }, "like_cnt": 0, "is_liked": -1 }
-
3-10. Update Post Comment
-
Endpoint:
/post/comment/{comment_id}
-
Request (
PATCH
){ "content": "string" }
-
Response
-
Status:
200 OK
{ "post_id": "string", "content": "string", "id": "string", "created_at": "2023-11-30T07:19:50.995Z", "updated_at": "2023-11-30T07:19:50.995Z", "user": { "id": "string", "username": "string", "profile_image_url": "string" }, "like_cnt": 0, "is_liked": -1 }
-
3-11. Delete Post Comment
-
Endpoint:
/post/comment/{comment_id}
-
Request(
DELETE
){}
-
Response
- Status:
200 OK
{ "message": "Comment {comment_id} deleted successfully" }
- Status:
3-12. Like Post
-
Endpoint:
/post/{post_id}/like
-
Request(
POST
){}
-
Response
- Status:
200 OK
{ "is_liked": 1, "user_id": "string", "created_at": "2023-11-30T07:31:44.298952+00:00", "id": "string", "post_id": "string", "updated_at": "2023-11-30T07:31:44.298952+00:00" }
- Status:
3-13. Like Comment
-
Endpoint:
/post/comment/{comment_id}/like
-
Request(
POST
){}
-
Response
- Status:
200 OK
{ "is_liked": 1, "user_id": "string", "created_at": "2023-11-30T07:31:44.298952+00:00", "id": "string", "comment_id": "string", "updated_at": "2023-11-30T07:31:44.298952+00:00" }
- Status:
4. Search
4-1. Search User
-
Endpoint:
/search/user/{search_string}
-
Request (
GET
){}
-
Response
-
Status:
200 OK
{ "total": 0, "items": [ { "email": "string", "id": "string", "username": "string", "profile_image_url": "string", "is_follower": bool, "is_following": bool } ], "next_cursor": 0 }
-
5. Notification
5-1. Get My Notifications
-
Endpoint:
/notification/me
-
Request (GET)
{}
-
Response
-
Status:
200 OK
{ "total": 0, "items": [ { "notification_type": "GENERAL", "read_at": "2023-11-19T13:14:07.939Z", "id": "string", "created_at": "2023-11-19T13:14:07.939Z", "updated_at": "2023-11-19T13:14:07.939Z", "sender_id": "string", "sender": { "id": "string", "username": "string", "profile_image_url": "string" }, "recipient_id": "string", "recipient": { "id": "string", "username": "string", "profile_image_url": "string" }, "post_id": "string" } ], "next_cursor": 0 }
-
5-2. Delete My All Notifications
-
Endpoint:
/notification/me
-
Request (DELETE)
{}
-
Response
-
Status:
200 OK
{ "message": "Deleted all notifications successfully" }
-
5-3. Delete My Specific Notifications
-
Endpoint:
/notification/{notification_id}
-
Request (DELETE)
{}
-
Response
-
Status:
200 OK
{ "message": "Deleted notification {notification_id} successfully" }
-
Name | Domain | Method | URL |
---|---|---|---|
Infer | stable_diffusion | POST | /models/stable_diffusion/infer |
Infer | open_seed | POST | /models/open_seed/infer |
1. Infer
1-1. stable_diffusion
-
Endpoint:
/models/stable_diffusion/infer
-
Request (
POST
){ "INPUT_IMAGE": "string", "MASK": "string", "PROMPT": "string", "NEGATIVE_PROMPT": "string", "SAMPLES": "INT32" }
-
Response
-
Status:
200 OK
{ "OUTPUT IMAGES": ["string",...] }
-
Status:
400 Bad Request
{ "error_code": 400, "message": "INVALID ARGS", }
-
1-2. open_seed
-
Endpoint:
/models/open_seed/infer
-
Request (
POST
){ "INPUT_IMAGE": "string", }
-
Response
-
Status:
200 OK
{ "OUTPUT_OVERALL_IMAGE": ["string"] "OUTPUT_MASKS": ["string",...] }
-
Above diagram demonstrates image generation framework used in our project. We utilized latent diffusion model SDXL with Progressive Adversarial Distillation for text guidance image generation and image conditional text guidance image inpainting.
For text guidance image generation, in latent space, distillation model maps unit gaussian noise distribution to data distribution with text conditioned and obtain output latent. And by forward passing it into Decoder we got final output image.
For image conditional text guidance image inpainting, we pass original image into Encoder to get latent and apply gaussian noise. And apply same schema as before to get output image and replace masked region of input image with output image.
https://huggingface.co/docs/diffusers/using-diffusers/sdxl
https://huggingface.co/ByteDance/SDXL-Lightning
OpenSeeD is a panoptic segmentation model that generates masks for each instance in input image which user can further modify this to specify the desired region to edit.
https://github.com/IDEA-Research/OpenSeeD
Frontend focuses on UI and Backend focuses on main logics
-
Setup the Testing Environment:
-
Use Espresso for UI testing
-
Add gradle dependencies as below
-
testImplementation 'junit:junit:4.13.2'
testImplementation "org.mockito:mockito-core:5.5.0"
testImplementation "androidx.arch.core:core-testing:2.2.0"
androidTestImplementation 'androidx.test.ext:junit:1.1.5'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.5.1'
androidTestImplementation 'androidx.test.espresso:espresso-intents:3.5.1'
androidTestImplementation platform('androidx.compose:compose-bom:2023.03.00')
androidTestImplementation 'androidx.compose.ui:ui-test-junit4'
debugImplementation 'androidx.compose.ui:ui-tooling'
debugImplementation 'androidx.compose.ui:ui-test-manifest'
-
Running Test
- Run 'Tests in 'com.project.spire''
-
Testing Result
- Passed 27 tests and tested on Nexus 5X, Galaxy S22 & S23
-
Setup the Testing Environment:
- Should install pytest, pytest-asyncio, pytest-cov packages
poetry install
- Organize Our Project Structure:
/spire-api
|-- main.py # FastAPI application
|-- /app
|-- __init__.py
|-- routers.py # FastAPI route handlers
|-- /tests
|-- __init__.py
|-- test_main.py # Test code
- Running Test
pytest --cov-report term-missing --cov
-
Testing Result
- Passed 78 tests and achieved 89% coverage (with 1,800+ lines of testing code)
---------- coverage: platform darwin, python 3.11.6-final-0 ----------
Name Stmts Miss Cover Missing
-------------------------------------------------------------------------
app/__init__.py 0 0 100%
app/api/api.py 19 0 100%
app/api/routers/auth.py 70 8 89% 96-100, 156, 172-175, 191
app/api/routers/image.py 27 8 70% 34-39, 55-62, 70-72
app/api/routers/notification.py 27 0 100%
app/api/routers/post.py 103 1 99% 287
app/api/routers/search.py 16 0 100%
app/api/routers/user.py 90 6 93% 42, 66-71, 86
app/core/__init__.py 0 0 100%
app/core/config.py 58 3 95% 73, 80, 117
app/core/conn.py 20 10 50% 11-12, 15-18, 21-24
app/core/exceptions/__init__.py 5 0 100%
app/core/exceptions/base.py 36 0 100%
app/core/exceptions/code.py 9 0 100%
app/core/exceptions/notification.py 5 0 100%
app/core/exceptions/post.py 17 0 100%
app/core/exceptions/token.py 9 0 100%
app/core/exceptions/user.py 37 0 100%
app/core/fastapi/__init__.py 0 0 100%
app/core/fastapi/dependency/__init__.py 2 0 100%
app/core/fastapi/dependency/permission.py 28 1 96% 14
app/core/fastapi/middleware/__init__.py 2 0 100%
app/core/fastapi/middleware/auth.py 38 11 71% 24-26, 29, 37-42, 44
app/core/fastapi/schema/__init__.py 0 0 100%
app/core/fastapi/schema/current_user.py 6 0 100%
app/main.py 46 14 70% 32-36, 43-47, 54-60
app/models/__init__.py 8 0 100%
app/models/base.py 9 0 100%
app/models/code.py 11 0 100%
app/models/guid.py 27 5 81% 18, 26-29, 36
app/models/notification.py 18 0 100%
app/models/post.py 58 0 100%
app/models/timestamp_mixin.py 12 0 100%
app/models/user.py 35 1 97% 13
app/repository/__init__.py 0 0 100%
app/repository/code.py 15 3 80% 16-18, 29
app/repository/comment.py 68 17 75% 67, 83, 88-133, 140, 147-148, 155, 165, 183-185, 190-195
app/repository/notification.py 51 7 86% 13, 58, 81-82, 93, 107, 118
app/repository/post.py 95 18 81% 12, 17, 142, 153, 161, 168-169, 176-177, 183, 191, 241, 251, 265-267, 272-277
app/repository/user.py 161 14 91% 28, 37, 61, 72-74, 81, 120, 128, 138, 183, 191, 200, 216-218
app/schemas/__init__.py 7 0 100%
app/schemas/auth.py 12 0 100%
app/schemas/code.py 4 0 100%
app/schemas/image.py 12 0 100%
app/schemas/jwt.py 4 0 100%
app/schemas/notification.py 39 0 100%
app/schemas/post.py 74 0 100%
app/schemas/user.py 49 0 100%
app/services/__init__.py 0 0 100%
app/services/auth_service.py 98 43 56% 36-57, 65-82, 87-92, 97-102, 107-108, 116-117, 175-180, 191-198
app/services/code_service.py 40 21 48% 17-36, 45-59
app/services/image_service.py 43 30 30% 13-25, 36-59, 68-106
app/services/jwt_service.py 16 3 81% 18-19, 24
app/services/notification_service.py 52 16 69% 25-26, 49-50, 68-72, 80-84, 92-96
app/services/post_service.py 174 30 83% 36-37, 51-52, 71-72, 100-101, 125-126, 129, 145, 148, 160-161, 172-173, 194-195, 241-242, 249-250, 256-257, 263-264, 300-302
app/services/user_service.py 131 35 73% 24-32, 52, 59, 68-69, 76, 83-84, 97, 120, 134, 147-148, 156-159, 165-172, 188-190, 201-202
app/session.py 31 1 97% 29
app/utils/__init__.py 0 0 100%
app/utils/aws.py 16 9 44% 19-35
app/utils/code_helper.py 5 0 100%
app/utils/common.py 9 0 100%
app/utils/ecs_log.py 18 0 100%
app/utils/json_decoder.py 27 5 81% 15-16, 35-37
app/utils/pagination.py 3 0 100%
app/utils/password_helper.py 9 0 100%
app/utils/token_helper.py 23 8 65% 28-31, 35-43
app/utils/user.py 4 0 100%
tests/__init__.py 0 0 100%
tests/test_api.py 802 0 100%
tests/test_base64.py 1 0 100%
tests/test_schema.py 36 0 100%
-------------------------------------------------------------------------
TOTAL 2977 328 89%
=================================================== 78 passed, 12 warnings in 12.05s ===================================================
We proceeded user acceptance testing for five key user stories as below.
- Given that the user is on the Spire login page, When the user selects "Sign up" and enters a valid email, password, and username, Then the user should be registered and logged into Spire with a new account.
- Given that the user is on the Spire search page, When the user enters a username into the search bar, Then users matching the search query should be displayed and the user can follow them.
- Given that the user is viewing a post, When they select "Like" or "Comment", Then their interaction should be recorded and displayed.
- Given that the user is in the image generation section of Spire, When they upload an image from gallery and apply a mask and a creative prompt, Then the image should be recreated according to the selected prompt.
- Given that the user is creating a new image from a text prompt, When they write some text prompt, Then the image should be generated.
Detailed steps and corresponding user acceptance criterion user are described on the following document. UAT_Team1