AECVision

AECVision is an object detection project for the construction industry. The main goal is to deliver good models for various problems like:

• Detection elements on plans to allow automatized tasks
• Speed up documentation analysis
• Create tools for future computer vision projects

Information

Now project is base on YOLOv5 repo but in the future I will try other types of model architectures.

• Input image resolution 1280x1280
• Image file .jpg
• For tagging i use Label Studio
• Training model in Google Colab
• Detection images with using Sahi

Folder structure

modules # Project modules
├───object_detection # Detection screen module
│   └───screen_detection.ipynb 
├───prepare_data_and_training # Prepare data and training in google colab module
│   ├───dataset
│   │   ├───images
│   │   │   ├───test
│   │   │   ├───training
│   │   │   └───validation
│   │   └───labels
│   │       ├───labels_reduce_classes
│   │       ├───test
│   │       ├───training
│   │       └───validation
│   ├───prepare_data_from_pdf.ipynb
│   └───train_model_in_google_colab.ipynb
├───wall_detection_export # Export walls detection coordinates to csv and allow imports in other program
│   ├───files
│   │   ├───converted_pdf
│   │   ├───exported_csv
│   │   └───upload_pdf
│   ├───classes_functions.py
│   ├───wall_detection_export_with_sahi.py
│   └───wall_detection_export.py
train_results # Available modules
├───model_12classes
├───model_12classes_gray
├───model_walls
└───model_walls_gray
yolov5 # Cloned Yolov5 repository
LICENSE
README.md
requirements.txt

Project pipeline

Available models

Name	Classes	Model architecture	Number of training images [original/augmented]	Color
model_12classes	12 Classes (see train_results)	YOLOv5m6	252 / 1204	RGB
model_walls	Wall	YOLOv5m6	252 / 1204	RGB
model_12classes_gray	12 Classes (see train_results)	YOLOv5m6	252 / 1204	GRAY
model_walls_gray	Wall	YOLOv5m6	252 / 1204	GRAY

How it works?

The model was trained on clear plans and with annotation but remember that the best results you get without many symbols on construction plans.

Detection

Confidence = 0.5

Confidence = 0.8

Screen detection

By using screen_detection.py

2023-06-18.13-26-16.mp4

Objects import from detections

Press image to watch film. This works with my PYLAB extension

Project roadmap

• Collect and tag more images (up to 500)
• Try to use Grayscale than RGB
• Change number of classes (maybe only 3? Wall, Window, Door)
• Evolve hyperparameters to get the best set (in YOLO)
• Use other model architecture (than YOLO)
• Think about switch from .jpg file to .png (avoid converting data lost)
• Think about how to add more "contexts" to the model (like: you are in the kitchen if there are stove and there probably will be table)

Summary & Problems

First training

Data description

Walls are overrepresented because this is normal quantities in architectural plans.

Maybe in future, i need to create more plans with other categories to improve other classes detection.

Learning process

Model has steadily improved up to the 272nd epoch and probably has further potential.

On the validation set, an increase in "obj_loss" was noted during the learning process.

Confussion matrix show that all classes except "background" are correctly identified.

If you want to dig into training results see train_results or write to me.

Problems

• A small number of certain classes
• Conversion from pdf (default construction plans format) to jpg results in a loss of image quality

Contributing

Contributions are always welcome! I look for help in tagging and improving models. Feel free to improve this project.

Thanks

Thanks, YOLOv5 and Label Studio for your project and tutorials. Special thanks to my brother Marcin who help me with tagging.