feat: Add GitHub Scanning app

[muntaxir4]

User description

Description

Created a Next.js app with GitHub scanner class under util to clone and use secret-scan on it.
Adds HTTP streaming response from API.

Related to #565

Screenshots of relevant screens

Developer's checklist

• My PR follows the style guidelines of this project
• I have performed a self-check on my work

If changes are made in the code:

• I have followed the coding guidelines
• My changes in code generate no new warnings
• My changes are breaking another fix/feature of the project
• I have added test cases to show that my feature works
• I have added relevant screenshots in my PR
• There are no UI/UX issues

Documentation Update

• This PR requires an update to the documentation at docs.keyshade.xyz
• [x I have made the necessary updates to the documentation, or no documentation changes are required.

---

PR Type

Enhancement, Tests

---

Description

• Added a GitHub repository scanner for secret detection, including support for .gitignore and binary file exclusions.

• Enhanced project management UI with updated API integration for secret and project handling.

• Refactored multiple services and components for improved performance, maintainability, and modularity.

• Introduced CLI commands for managing workspace memberships, including invitations, role updates, and ownership transfers.

• Added reusable UI components like Select and improved email templates using React Email.

• Expanded test coverage across schemas, APIs, and services with additional validations, edge cases, and modular test structures.

---

Changes walkthrough 📝

	Relevant files
Tests	17 files project.spec.ts Expanded schema test coverage with additional cases and validations. packages/schema/tests/project.spec.ts Added new test cases for schemas like ProjectWithCountSchema, CreateProjectRequestSchema, UpdateProjectRequestSchema, and others. Introduced validations for invalid inputs and missing fields across multiple schemas. Enhanced test coverage for edge cases and schema-specific constraints. Organized tests into separate describe blocks for better readability and structure. +553/-476 secret.spec.ts Enhanced secret schema tests with additional validations. packages/schema/tests/secret.spec.ts Added new test cases for SecretSchema, CreateSecretRequestSchema, UpdateSecretResponseSchema, and others. Improved validation checks for invalid inputs, missing fields, and incorrect data types. Organized tests into describe blocks for better modularity and clarity. Enhanced test coverage for schema-specific constraints and edge cases. +430/-413 workspace.spec.ts Improved workspace schema tests with additional cases and validations. packages/schema/tests/workspace.spec.ts Added new test cases for workspace-related schemas like InviteMemberRequestSchema, CreateWorkspaceRequestSchema, and others. Improved validation checks for invalid inputs, missing fields, and incorrect data types. Organized tests into describe blocks for better modularity and clarity. Enhanced test coverage for schema-specific constraints and edge cases. +410/-378 user.spec.ts Expanded user schema tests with additional validations and cases. packages/schema/tests/user.spec.ts Added new test cases for UserSchema, GetSelfResponseSchema, UpdateSelfRequestSchema, and others. Improved validation checks for invalid inputs, missing fields, and incorrect data types. Organized tests into describe blocks for better modularity and clarity. Enhanced test coverage for schema-specific constraints and edge cases. +211/-135 variable.e2e.spec.ts Added tests for variable versioning and metadata validation. apps/api/src/variable/variable.e2e.spec.ts Added new test case to validate fetching only new versions of variables. Enhanced metadata validation for variable-related API responses. Improved test coverage for variable updates and versioning scenarios. +81/-2    integration.spec.ts Refactor and enhance integration schema test cases.            packages/schema/tests/integration.spec.ts Grouped tests into descriptive describe blocks for better organization. Added missing expect statements for validation results in tests. Enhanced test coverage by including edge cases and optional fields. +326/-315 workspace-role.spec.ts Refactor and expand workspace role schema tests.                  packages/schema/tests/workspace-role.spec.ts Organized tests into describe blocks for clarity. Improved test coverage with additional cases for optional and invalid inputs. Added expect statements to validate test outcomes. +341/-330 variable.spec.ts Refactor and enhance variable schema test cases.                  packages/schema/tests/variable.spec.ts Grouped tests into describe blocks for better readability. Added validations for optional fields and edge cases. Improved error handling tests with detailed expect statements. +363/-352 api-key.spec.ts Refactor and improve API key schema tests.                              packages/schema/tests/api-key.spec.ts Organized API key schema tests into describe blocks. Enhanced test coverage for optional fields and invalid inputs. Added detailed expect statements for validation results. +280/-267 event.spec.ts Refactor and enhance event schema test cases.                        packages/schema/tests/event.spec.ts Grouped event schema tests into describe blocks for clarity. Added validations for optional fields and edge cases. Improved error handling tests with detailed expect statements. +110/-108 workspace-membership.e2e.spec.ts Update workspace membership e2e tests with new validations. apps/api/src/workspace-membership/workspace-membership.e2e.spec.ts Added createdOn field validation in workspace membership tests. Updated payload structure for user removal API tests. Enhanced test coverage for workspace membership scenarios. +10/-4    environment.spec.ts Refactor environment schema tests for better organization. packages/schema/tests/environment.spec.ts Refactored test structure by grouping related tests into describe blocks. Added more descriptive test organization for schema validation. Improved readability and modularity of test cases. +265/-254 workspace-membership.spec.ts Add workspace membership schema tests.                                      packages/schema/tests/workspace-membership.spec.ts Added comprehensive test cases for workspace membership schemas. Validated various scenarios for schema correctness and error handling. Ensured coverage for all workspace membership-related operations. +452/-0  pagination.spec.ts Refactor pagination schema tests for clarity.                        packages/schema/tests/pagination.spec.ts Refactored pagination schema tests into grouped describe blocks. Added more detailed validation for pagination metadata and errors. Improved test readability and modularity. +150/-146 enums.test.ts Refactor and enhance enum tests.                                                  packages/schema/tests/enums.test.ts Organized enum tests into grouped describe blocks. Ensured all enums have correct values and are tested thoroughly. Improved test structure and readability. +136/-118 workspace.e2e.spec.ts Add workspace invitation and membership tests.                      apps/api/src/workspace/workspace.e2e.spec.ts Added tests for fetching workspace invitations. Verified scenarios for accepted invitations and ownership transfers. Enhanced test coverage for workspace membership features. +110/-2  secret.e2e.spec.ts Add and enhance secret controller tests.                                  apps/api/src/secret/secret.e2e.spec.ts Enhanced secret fetching tests to include versioning and decryption scenarios. Added validation for metadata and response structure. Improved test coverage for secret-related operations. +101/-10
Enhancement	10 files workspace.command.ts Add workspace membership command to CLI.                                  apps/cli/src/commands/workspace.command.ts Added WorkspaceMembershipCommand to the list of workspace commands. Improved modularity by including membership-related functionality. +3/-1      page.tsx Update project management UI and integrate new API.            apps/platform/src/app/(main)/page.tsx Enhanced project creation flow with new dialog and form fields. Integrated project fetching and creation with updated API controllers. Improved UI/UX for project management in the dashboard. +234/-87 select.tsx Add reusable Select component with Radix UI.                          apps/platform/src/components/ui/select.tsx Added a reusable Select component using Radix UI primitives. Included styling and utility functions for dropdown functionality. Enhanced accessibility and customization options for select inputs. +158/-0  project.service.ts Refactor project service to include counts and modularize logic. apps/api/src/project/service/project.service.ts Refactored project data fetching to include environment, variable, and secret counts. Added a helper function to modularize counting logic. Improved performance and maintainability of project-related queries. +76/-69  scan-repo.ts Add GitHub repository scanner for secret detection.            apps/github-scanner/src/util/scan-repo.ts Implemented a GitHub repository scanner for detecting secrets. Added functionality to clone repositories and scan files for sensitive data. Included support for .gitignore and binary file exclusions. +192/-0  workspace-removal.tsx Refactor workspace removal email template.                              apps/api/src/mail/emails/workspace-removal.tsx Refactored workspace removal email template to use a base template. Simplified email structure and centralized styling. Improved maintainability and consistency of email templates. +21/-130 secret.service.ts Refactor secret service for optimized version handling.    apps/api/src/secret/service/secret.service.ts Refactored secret fetching logic to optimize version selection. Improved handling of environment-specific secret versions. Enhanced maintainability and performance of secret-related queries. +58/-77  page.tsx Update secret page with new API integration.                          apps/platform/src/app/(main)/project/[project]/@secret/page.tsx Updated secret page to use new API controllers for fetching secrets. Improved UI for displaying secret details and metadata. Enhanced loading state handling and error logging. +35/-28  mail.service.ts Refactor email service to use React Email templates.          apps/api/src/mail/services/mail.service.ts Refactored email service to use React Email templates. Added support for workspace invitation and OTP email templates. Improved email rendering and maintainability. +35/-61  membership.workspace.ts Add CLI commands for workspace membership management.        apps/cli/src/commands/workspace/membership.workspace.ts Added CLI commands for managing workspace memberships. Included support for invitations, role updates, and ownership transfers. Enhanced CLI functionality for workspace-related operations. +34/-0
Additional files	101 files deploy-api.yml +20/-24  deploy-platform.yml +20/-24  deploy-web.yml +36/-40  validate-api-client.yaml +10/-2    validate-api.yaml +2/-2      validate-cli.yaml +8/-4      validate-platform.yaml +7/-5      validate-schema.yaml +2/-2      validate-secret-scan.yaml +2/-2      validate-web.yaml +4/-4      CHANGELOG.md +43/-0    Logout.bru +18/-0    Get all invitations of user.bru +21/-0    Remove users from workspace.bru +14/-0    package-lock.json +2111/-59 auth.controller.ts +6/-0      auth.service.ts +12/-0    create.environment.ts +2/-1      environment.e2e.spec.ts +19/-1    base-email-template.tsx +84/-0    otp-email-template.tsx +34/-0    common-styles.ts +86/-0    workspace-invitation.tsx +64/-0    interface.service.ts +4/-3      mock.service.ts +6/-5      migration.sql +2/-0      schema.prisma +1/-0      project.e2e.spec.ts +34/-6    project.types.ts +6/-0      variable.service.ts +56/-70  workspace-membership.controller.ts +2/-2      workspace-membership.service.ts +7/-2      workspace.controller.ts +20/-0    workspace.service.ts +105/-0  accept-invitation.membership.ts +49/-0    cancel-invitation.membership.ts +55/-0    decline-invitation.membership.ts +49/-0    get-all-members.membership.ts +64/-0    invite.membership.ts +65/-0    leave.membership.ts +49/-0    remove.membership.ts +55/-0    transfer-ownership.membership copy.ts +55/-0    update-role.membership.ts +61/-0    update.role.ts +2/-2      controller-instance.ts +14/-1    .env.example +1/-0      .eslintrc.js +24/-0    README.md +3/-0      next.config.mjs +4/-0      package.json +29/-0    postcss.config.mjs +8/-0      route.ts +39/-0    globals.css +27/-0    layout.tsx +35/-0    page.tsx +12/-0    repo-input.tsx +109/-0  types.ts +16/-0    tailwind.config.ts +19/-0    tsconfig.json +13/-0    .eslintrc.cjs +6/-9      Dockerfile +2/-6      package.json +2/-1      layout.tsx +16/-8    page.tsx +33/-54  page.tsx +2/-4      page.tsx +3/-2      layout.tsx +5/-1      online-status-handler.tsx +10/-0    index.tsx +22/-33  index.tsx +4/-8      index.tsx +1/-1      combobox.tsx +52/-34  use-online-status.ts +33/-0    api-client.ts +0/-93    projects.ts +0/-77    secrets.ts +0/-26    controller-instance.ts +72/-12  workspace-storage.ts +1/-1      index.ts +0/-99    tsconfig.json +13/-2    Dockerfile +1/-3      index.tsx +3/-3      api-testing.md +9/-0      erDetails(userData) +664/-0  package.json +9/-5      project.ts +9/-9      workspace-membership.ts +2/-2      client.ts +6/-1      index.ts +3/-1      workspace-membership.types.d.ts +0/-145  setup.ts +1/-1      workspace-membership.spec.ts +1/-1      package-lock.json +1239/-172 package.json +9/-9      index.ts +1/-0      index.types.ts +1/-0      index.ts +17/-3    index.types.ts +4/-1      index.ts +12/-10  index.ts +7/-9      Additional files not shown

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[codiumai-pr-agent-free]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 565 - Partially compliant Compliant requirements: Created a new Next.js app under 'apps' directory Added GitHub scanner class to scan repositories Uses secret-scan package for scanning Non-compliant requirements: No implementation shown for displaying scan results (files, lines, secret types) Requires further human verification: Verify the scanning functionality works correctly with real GitHub repos Check if the UI properly displays all scan results Test error handling for invalid repo URLs

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review API Change Changed removeUsers API to accept comma-separated emails as query param instead of body array. This could break existing API clients. async removeUsers( @CurrentUser() user: User, @Param('workspaceSlug') workspaceSlug: Workspace['slug'], @Query('userEmails') userEmails: string = '' ) { return this.workspaceMembershipService.removeUsersFromWorkspace( user, workspaceSlug, userEmails.split(/\s*,\s*/) ) }

[codiumai-pr-agent-free]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Score
Possible issue	Fix incorrect type validation test Fix inconsistent type validation in RollBackSecretResponseSchema test - the schema expects count to be a number but test validates it as string packages/schema/tests/secret.spec.ts [293-298] it('should validate a valid RollBackSecretResponseSchema', () => { const result = RollBackSecretResponseSchema.safeParse({ - count: '1' + count: 1 }) expect(result.success).toBe(true) }) Apply this suggestion Suggestion importance[1-10]: 9 Why: The test is incorrectly validating a string value ('1') for the count field when the schema expects a number, which would cause the test to fail. This is a critical bug fix in the test suite.	9
	Fix incorrect data structure test Fix incorrect values structure in GetAllSecretsOfProjectResponseSchema test - values should be an array but is defined as an object packages/schema/tests/secret.spec.ts [352-361] -values: { +values: [{ environment: { id: 'env123', name: 'Development', slug: 'development' }, value: 'secret-value', version: 1 -} +}] Apply this suggestion Suggestion importance[1-10]: 9 Why: The test uses an incorrect data structure for the 'values' field (object instead of array), which would cause schema validation to fail. This is a critical fix for test accuracy.	9
General	Add missing schema validation test Add test case to verify that ProjectSchema rejects when forkedFromId is provided but isForked is false, since this is an invalid state that should be prevented. packages/schema/tests/project.spec.ts [61-84] it('should not validate ProjectSchema when isForked is true but forkedFromId is null', () => { const result = ProjectSchema.safeParse({ id: 'project123', name: 'Project Name', slug: 'project-slug', description: 'Project Description', createdAt: '2024-10-01T00:00:00Z', updatedAt: '2024-10-01T00:00:00Z', publicKey: 'public-key', privateKey: 'private-key', storePrivateKey: true, isDisabled: false, accessLevel: 'GLOBAL', pendingCreation: false, isForked: true, lastUpdatedById: 'user123', workspaceId: 'workspace123', forkedFromId: null }) + + expect(result.success).toBe(false) + expect(result.error?.issues).toHaveLength(1) + expect(result.error?.issues[0].message).toBe('Invalid input') }) +it('should not validate ProjectSchema when forkedFromId is provided but isForked is false', () => { + const result = ProjectSchema.safeParse({ + id: 'project123', + name: 'Project Name', + slug: 'project-slug', + description: 'Project Description', + createdAt: '2024-10-01T00:00:00Z', + updatedAt: '2024-10-01T00:00:00Z', + publicKey: 'public-key', + privateKey: 'private-key', + storePrivateKey: true, + isDisabled: false, + accessLevel: 'GLOBAL', + pendingCreation: false, + isForked: false, + lastUpdatedById: 'user123', + workspaceId: 'workspace123', + forkedFromId: 'parent123' + }) + + expect(result.success).toBe(false) + expect(result.error?.issues).toHaveLength(1) + expect(result.error?.issues[0].message).toBe('Invalid input') +}) + Apply this suggestion Suggestion importance[1-10]: 8 Why: The suggestion adds an important test case to verify the schema validation for an invalid state where forkedFromId is provided but isForked is false, complementing the existing test for the opposite case. This helps ensure data integrity and schema correctness.	8
	Add empty array validation test Add test cases to verify that the roleSlugs array cannot be empty when provided in InviteMemberRequestSchema. Currently there are no tests checking this important validation. packages/schema/tests/workspace.spec.ts [22-28] it('should validate if proper input is specified for InviteMemberRequestSchema', () => { const result = InviteMemberRequestSchema.safeParse({ email: '[email protected]', roleSlugs: ['role1', 'role2'] }) - expect(result.success).toBe(true) }) +it('should not validate if roleSlugs array is empty', () => { + const result = InviteMemberRequestSchema.safeParse({ + email: '[email protected]', + roleSlugs: [] + }) + expect(result.success).toBe(false) + expect(result.error?.issues[0].path).toEqual(['roleSlugs']) +}) + Apply this suggestion Suggestion importance[1-10]: 7 Why: The suggestion adds an important validation test case to verify that empty roleSlugs arrays are rejected, which is a critical validation check missing from the current test suite. This helps ensure data integrity and proper role assignment validation.	7

[rajdip-b]

Looks like we have a lot of conflicts!