FJob backend

FJob is a fullstack application (main technologies: Django Rest Framework, Nextjs, Selenium). FJob, by collecting job offers from many job portals (currently only JustJoinIT, but there are already scripts ready to collect data from: olx, Pracujpl, nofluffjobs, Pracujpl) and enabling companies to post their own job offers will allow you to easily search for job offers.

Report Bug · Request Feature

!Info!

The project has undergone significant changes in terms of code architecture. Initially, all application logic resided in views. Currently, thanks to the use of Repository Pattern and Service Layer, views have been significantly improved. Not all views use this architecture, I left some of them unchanged because they use abstract classes from DRF, so there is practically no business logic in them.

---

The application is no longer available in the demo version, I don't have money to maintain it with this architecture :(

System Architecture

Fjob consists of 3 repositories:

• Backend (current)
• Frontend https://github.com/DEENUU1/fjob_frontend
• Scrapers https://github.com/DEENUU1/fjob_scrapers

Features

!NEW! I started refactoring the current application and moved a large part of the code initially saved in views to the service layer using the repository pattern. Some views have remained unchanged due to, for example, their simplicity related to the use of Django

1. JWT and OAuth Google Integration:
   • Utilizes JSON Web Tokens (JWT) in conjunction with OAuth Google for secure authentication and authorization processes.
2. Web Scraping and API-driven Database Storage:
   • Implements web scraping techniques to extract job offers from various websites.
   • Processes and transmits the gathered data via API for storage in the database.
3. Error Reporting and Favorites Functionality:
   • Incorporates a robust error reporting mechanism.
   • Enables users to save job offers to favorites for convenient retrieval.
4. Advanced Filtering, Sorting, Pagination, and Search Capabilities:
   • Provides sophisticated filtering, sorting, and pagination options for efficient data exploration.
   • Facilitates advanced job search functionalities.
5. Stripe Integration for Business Listings Purchase:
   • Integrates with Stripe payment gateway, allowing businesses to purchase listings seamlessly.
6. User and Company Account Management:
   • Enables the creation of both regular user accounts and company accounts.
7. Django Signals for Automated Services:
   • Utilizes Django Signals for the automation of various services, including updating available job offers, sending email notifications post-payment, and handling errors in job offers.
   • Instantiates Company objects following the creation of a company account.
8. Custom Commands and Permissions:
   • Implements custom commands and permissions for enhanced control and flexibility.
9. Default Data Upload and Localization:
   • Includes functionality for uploading default countries, cities, and regions (currently focused on Poland).
10. NextJS and Typescript for Frontend:
    • Develops the frontend application using NextJS and Typescript for a robust and type-safe user interface.
11. CI/CD with GitHub Actions:
    • Implements Continuous Integration/Continuous Deployment (CI/CD) using GitHub actions.
    • Automates unit tests and deploys the latest code version to an AWS EC2 instance.
12. NGINX Configuration for Network Traffic Handling:
    • Utilizes NGINX on the EC2 instance to efficiently manage network traffic.
13. SSL Certificate Renewal with Certbot:
    • Configures Certbot for automatic SSL certificate renewal, ensuring secure communication.
14. Background Task Handling with Redis and Celery:
    • Incorporates Redis and Celery for background task processing.
15. AWS Integration for Scalability and Reliability:
    • Integrates Django Rest Framework with AWS S3 Bucket for storing static files and model data.
    • Utilizes Google Cloud SQL for a reliable and secure database.
    • Leverages AWS services such as CodeBuild for web scraping functions on Lambda, ElasticIP Addresses for fixed IP assignment, Route 53 for domain management, and Amplify for hosting and deployment.
16. Event-Driven Architecture with AWS EventBridge:
    • Implements AWS EventBridge to automatically trigger each web scraping function.
17. Sentry Integration for Error Monitoring:
    • Integrates with the Sentry platform to monitor and analyze errors in the system.
18. I wrote over 120 tests, both unit and integration

Technologies:

• Python
  • Django / Django Rest Framework
  • Selenium
  • Celery
  • Pytest
  • Requests
  • Beautiful Soup 4
• React
  • Next.JS
  • Typescript
• AWS
  • EC2
  • S3 Bucket
  • Elastic IP
  • Route 53
  • Lambda
  • EventBridge
  • CodeBuild
  • Amplify
• Google Cloud SQL (postgreSQL)
• Docker
• Docker Compose
• Redis
• CI/CD
• Certbot
• NGINX
• SQLite
• Stripe

Backend - Endpoints

Frontend

I'm not a frontend developer, but I think it looks pretty good

Local environment installation

Backend in local mode use SQLite so there is no need to use Docker/Docker-compose

Installation - without docker

# Clone repository
git clone <url>

# Create .env file 
cp .env_example .env

# Install libraries
pip install -r requirements.txt

# Apply migrations
python manage.py migrate 

# Create super user 
python manage.py createsuperuser (in User Type type "USER")

# Run server 
python manage.py runserver 

Installation - with docker

# Clone repository
git clone <url>

# Create .env file 
cp .env_example .env

# Build image
docker-compose build

# Run containers
docker-compose up # -d 

# Apply migrations
python manage.py migrate 

# 
# Run this commands inside web container
# 

# Create super user 
python manage.py createsuperuser (in User Type type "USER")

# Run server 
python manage.py runserver 

Commands

Run pytest

pytest 

Create default Product objects

python manage.py products

Upload default cities and regions for Poland

python manage.py poland 

Create default CompanyCategory objects

python manage.py category

Stripe

To run stripe locally use this command to run a webhook

stripe listen --forward-to localhost:8000/api/payment/new_company/webhook

Authors

• @DEENUU1

License

See LICENSE.txt for more information.