Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V3.0.0 #9

Merged
merged 133 commits into from
Feb 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
133 commits
Select commit Hold shift + click to select a range
40ef28d
Configure and dd dev-environment
Feb 2, 2024
7f4146e
try to get testing to work
Feb 2, 2024
774ff06
testfile
Feb 2, 2024
4147ab1
add workflow_dispatch to action
Feb 2, 2024
c437db7
Merge pull request #5 from danpejobo/test
danpejobo Feb 2, 2024
83cc5ac
add pycodestyle to makefile
Feb 2, 2024
096e0c6
remove qgis 3_22
Feb 2, 2024
5f8e829
Merge pull request #6 from danpejobo/test
danpejobo Feb 2, 2024
763fa5f
trying tests again
Feb 2, 2024
51e0915
test
Feb 2, 2024
ec3609e
update action
Feb 2, 2024
d1186dc
try another pth
Feb 2, 2024
194fc6c
update path in github action
Feb 2, 2024
a2e8a4b
edit path
Feb 2, 2024
cb5519a
edit variable
Feb 2, 2024
bb7fa6d
add debug step
Feb 2, 2024
ce37e9e
add directory
Feb 2, 2024
bf45441
edit paths
Feb 2, 2024
ed53a2d
edit paths
Feb 2, 2024
55f45f9
update import statement and tests
Feb 2, 2024
470efb6
update tests
Feb 2, 2024
ae635c4
add _init__ to submodule in github action
Feb 2, 2024
4b92c6d
edit path
Feb 2, 2024
bc1a05e
update test paths
Feb 2, 2024
b52342b
edit paths
Feb 2, 2024
93977e3
add submodule
Feb 2, 2024
f2aa839
tests update
Feb 2, 2024
f52fd37
update test
Feb 2, 2024
ebf587a
edit test
Feb 2, 2024
ae124c9
tests
Feb 3, 2024
3681764
test import error
Feb 3, 2024
5651464
update tests
Feb 3, 2024
4e91c9a
edit test
Feb 3, 2024
c8ca6e9
update test
Feb 3, 2024
87c5072
test case
Feb 3, 2024
e8eff50
short only
Feb 3, 2024
c3a3783
test case
Feb 3, 2024
32bcce8
run test on main/dev on push/pull-req
Feb 3, 2024
bc3168f
add test calls
Feb 3, 2024
14db881
error in yml file
Feb 3, 2024
9f9cd84
edit tests
Feb 3, 2024
1f53ab0
update testcase
Feb 3, 2024
e6ea2af
edit test case
Feb 3, 2024
badfe26
try cp testfiles
Feb 3, 2024
3e99a5c
edit steps
Feb 3, 2024
f8a5cb1
edit test and settings
Feb 4, 2024
1e7837b
edit test
Feb 4, 2024
fb2942c
update test
Feb 4, 2024
5b2fb09
restructure repo
Feb 4, 2024
f538b13
restructure repo
Feb 4, 2024
3b16c94
update readme
Feb 4, 2024
c0096ab
update readme
Feb 4, 2024
ab116e8
edit readme
Feb 5, 2024
e21ec3e
update readme
Feb 5, 2024
68e5ada
edit makefile to reflect new repo structure
Feb 5, 2024
393371e
update paths in github actions
Feb 5, 2024
9e236d3
edit action and test
Feb 5, 2024
9579b04
change name on utils to utilities
Feb 5, 2024
73fe1ed
edit github action
Feb 5, 2024
4407f5f
edit check plugin load
Feb 5, 2024
f301f7c
edit tests, add badge to readme
Feb 6, 2024
c85579e
edit tests
Feb 6, 2024
9e70092
update test
Feb 6, 2024
db1d600
try to fix import. relative to absolute
Feb 6, 2024
0f9669e
edit docstring
Feb 6, 2024
564f364
update docstring
Feb 6, 2024
e98c688
update docstring
Feb 6, 2024
6c42d5c
update test
Feb 6, 2024
dedd2f7
edit test
Feb 6, 2024
43330a6
update test
Feb 6, 2024
f6093cb
update test
Feb 6, 2024
5a5ccf8
update test
Feb 6, 2024
37865a7
update test
Feb 6, 2024
683cc1d
update test
Feb 6, 2024
9d1f292
update testing
Feb 6, 2024
a3bd753
update test
Feb 6, 2024
aa69c0a
test update
Feb 6, 2024
428dd6f
update test, add feedback if exception
Feb 6, 2024
0b47615
change to absolute import
Feb 7, 2024
6aff859
edit test
Feb 7, 2024
e3e2c1e
edit test
Feb 7, 2024
96035a5
Merge pull request #7 from danpejobo/restructure-repo
danpejobo Feb 7, 2024
85208b6
update text for output feature name variable
Feb 7, 2024
d525984
update test
Feb 7, 2024
1537750
update test and add test badge to readme
Feb 7, 2024
fc2e001
add issue templates
Feb 7, 2024
8f6c7ed
update test
Feb 7, 2024
70644cb
change testcase CRS
Feb 7, 2024
c6c151d
exclude long-term from test
Feb 7, 2024
be54e53
update test
Feb 7, 2024
3a7d472
remove redundant logger
Feb 7, 2024
ea36d35
edit util reproject raster "TARGET_CRS"
Feb 7, 2024
c91e67b
edit test
Feb 7, 2024
8131fcd
edit test
Feb 7, 2024
650cd78
add context to temp folder
Feb 7, 2024
f663ca7
add prefix to logging and feedback in @reproject_layers
Feb 8, 2024
abfc216
try to use postprocessing
Feb 8, 2024
3fd2693
edit test
Feb 8, 2024
c0505a2
remove intermediate option
Feb 8, 2024
fcf3156
remove intermediate laayers #2
Feb 8, 2024
be4e706
edit algorithm to postprocess
Feb 8, 2024
32fa244
add QgsProcessingException
Feb 8, 2024
424b465
edit param input for reproject
Feb 8, 2024
d46d8a7
edit reproject
Feb 8, 2024
928a78e
edit begrensskade
Feb 8, 2024
9020f4e
update test
Feb 8, 2024
8d57734
edit post-process
Feb 8, 2024
74c47ad
try to edit the addlayerstask
Feb 8, 2024
b1a5899
edit test and log values to reproject
Feb 8, 2024
9286ee6
edit reproject
Feb 8, 2024
a29fb5b
is child added
Feb 8, 2024
4df7bfe
try with reproject raster change input and source_crs
Feb 8, 2024
34e1993
try postprocess with addlayerstask class
Feb 9, 2024
b00af74
add tests and test data
Feb 9, 2024
74ad0f3
format code
Feb 9, 2024
9def931
Merge branch 'dev' of https://github.com/danpejobo/geovita_processing…
Feb 9, 2024
cfbd521
edit add layers class
Feb 9, 2024
9075e26
Merge pull request #8 from danpejobo/postprocessing
danpejobo Feb 9, 2024
c89b692
add docstring to postprocess
Feb 9, 2024
a74ba12
edit tests for excavation module
Feb 9, 2024
3afc969
update testcase
Feb 9, 2024
e7c757c
remove tricky "lower()" call to raster path.
Feb 9, 2024
2d16b60
edit wrong testnumber
Feb 9, 2024
9ff7479
add tests to impact map
Feb 9, 2024
a040570
edit tests
Feb 9, 2024
9283224
update test. error in calling alg.
Feb 9, 2024
024c9b5
update parameters - test for tunnel
Feb 9, 2024
6d4620a
update readme
Feb 9, 2024
fa62405
update readme
Feb 9, 2024
0f126fe
update testcase
Feb 9, 2024
19dc4d3
edit copyright dates
Feb 9, 2024
a80c3f4
edit sub-processes parameters
Feb 9, 2024
6eb9ec4
v.3.0.0
Feb 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is. If you hit some error please paste the entire Python stacktrace message.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error


**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- QGIS release [e.g. QGIS 3.28]
- geovita_processing_provider release [e.g. 2.0.2]

**Additional context**
Add any other context about the problem here.
17 changes: 17 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: ''

---

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots and URL about the feature request here.
97 changes: 97 additions & 0 deletions .github/workflows/test_plugin.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
name: Test plugin

on:
push:
branches:
- main
- dev
paths:
- "geovita_processing_plugin/**"
- ".github/workflows/test_plugin.yml"
pull_request:
branches:
- main
- dev
paths:
- "geovita_processing_plugin/**"
- ".github/workflows/test_plugin.yml"
workflow_dispatch:

env:
# plugin name/directory where the code for the plugin is stored
PLUGIN_NAME: geovita_processing_plugin
# python notation to test running inside plugin
TESTS_RUN_FUNCTION: geovita_processing_plugin.test_suite.test_package
# Docker settings
DOCKER_IMAGE: qgis/qgis

jobs:

Test-geovita_processing_plugin:

runs-on: ubuntu-latest

strategy:
matrix:
docker_tags: [release-3_28, latest]

steps:

- name: Checkout
uses: actions/checkout@v4
with:
submodules: 'recursive'

- name: Docker pull and create qgis-testing-environment
run: |
docker pull "$DOCKER_IMAGE":${{ matrix.docker_tags }}
docker run -d -e XDG_RUNTIME_DIR=/tmp/runtime-root --name qgis-testing-environment -v ${{ github.workspace }}:/tests_directory -e DISPLAY=:99 "$DOCKER_IMAGE":${{ matrix.docker_tags }}

- name: List mounted directory contents
run: |
docker exec qgis-testing-environment ls -la /tests_directory
docker exec qgis-testing-environment ls -la /tests_directory/$PLUGIN_NAME

- name: Docker set up QGIS
run: |
docker exec qgis-testing-environment sh -c "qgis_setup.sh $PLUGIN_NAME"
docker exec qgis-testing-environment sh -c "cat /root/.local/share/QGIS/QGIS3/profiles/default/QGIS/QGIS3.ini"
docker exec qgis-testing-environment sh -c "rm -f /root/.local/share/QGIS/QGIS3/profiles/default/python/plugins/$PLUGIN_NAME"
docker exec qgis-testing-environment sh -c "ln -s /tests_directory/$PLUGIN_NAME /root/.local/share/QGIS/QGIS3/profiles/default/python/plugins/$PLUGIN_NAME"
docker exec qgis-testing-environment sh -c "pip3 install -r /tests_directory/REQUIREMENTS_TESTING.txt"

docker exec qgis-testing-environment sh -c "ls -l /root/.local/share/QGIS/QGIS3/profiles/default/python/plugins/"
docker exec qgis-testing-environment sh -c "ls -l /root/.local/share/QGIS/QGIS3/profiles/default/python/plugins/$PLUGIN_NAME"

- name: Prepare REMEDY_GIS_RiskTool submodule
run: |
docker exec qgis-testing-environment sh -c "touch /tests_directory/$PLUGIN_NAME/REMEDY_GIS_RiskTool/__init__.py"
docker exec qgis-testing-environment ls -la /tests_directory/$PLUGIN_NAME/REMEDY_GIS_RiskTool

- name: Docker run plugin tests
run: |
docker exec qgis-testing-environment sh -c "export PYTHONPATH=/root/.local/share/QGIS/QGIS3/profiles/default/python/plugins:$PYTHONPATH && qgis_testrunner.sh $TESTS_RUN_FUNCTION"

Check-code-quality:
runs-on: ubuntu-latest
steps:

- name: Install Python
uses: actions/setup-python@v5
with:
python-version: '3.9'
architecture: 'x64'

- name: Checkout
uses: actions/checkout@v4

- name: Install packages
run: |
pip install -r REQUIREMENTS_TESTING.txt
pip install pylint pycodestyle

- name: Pylint
run: make pylint

- name: Pycodestyle
run: make pycodestyle
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -155,4 +155,7 @@ cython_debug/
zip_build/

#VSCode stuff
.vscode/
.vscode/

#Ruff
.ruff*
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
[submodule "REMEDY_GIS_RiskTool"]
path = REMEDY_GIS_RiskTool
url = https://github.com/norwegian-geotechnical-institute/REMEDY_GIS_RiskTool.git
[submodule "geovita_processing_plugin/REMEDY_GIS_RiskTool"]
path = geovita_processing_plugin/REMEDY_GIS_RiskTool
url = https://github.com/norwegian-geotechnical-institute/REMEDY_GIS_RiskTool.git
12 changes: 6 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ PEP8EXCLUDE=pydev,resources.py,conf.py,third_party,ui
# * Windows:
# AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins'

QGISDIR=C:\Users\dpe\AppData/Roaming/QGIS/QGIS3/profiles/default/python/plugins
QGISDIR=.local/share/QGIS/QGIS3/profiles/default/python/plugins/

#################################################
# Normally you would not need to edit below here
Expand Down Expand Up @@ -222,7 +222,7 @@ pylint:
@echo "-----------------"
@echo "Pylint violations"
@echo "-----------------"
@pylint --reports=n --rcfile=pylintrc . || true
@pylint --reports=n --rcfile=pylintrc geovita_processing_plugin || true
@echo
@echo "----------------------"
@echo "If you get a 'no module named qgis.core' error, try sourcing"
Expand All @@ -231,14 +231,14 @@ pylint:
@echo "----------------------"


# Run pep8 style checking
# Run pep8/pycodestyle style checking
#http://pypi.python.org/pypi/pep8
pep8:
pycodestyle:
@echo
@echo "-----------"
@echo "PEP8 issues"
@echo "pycodestyle PEP8 issues"
@echo "-----------"
@pep8 --repeat --ignore=E203,E121,E122,E123,E124,E125,E126,E127,E128 --exclude $(PEP8EXCLUDE) . || true
@pycodestyle --repeat --ignore=E203,E121,E122,E123,E124,E125,E126,E127,E128 --exclude $(PEP8EXCLUDE) geovita_processing_plugin || true
@echo "-----------"
@echo "Ignored in PEP8 check:"
@echo $(PEP8EXCLUDE)
63 changes: 14 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,68 +1,33 @@
![Geovita Logo](icons/geovita.ico)
![Geovita Logo](geovita_processing_plugin/icons/geovita.ico)

# Geovita Processing Provider
Geovita Processing Provider
===========================
[![Static Badge](https://img.shields.io/badge/plugins.QGIS.org-published-green)](https://plugins.qgis.org/plugins/geovita_processing_plugin/)
[![Test plugin](https://github.com/danpejobo/geovita_processing_plugin/actions/workflows/test_plugin.yml/badge.svg)](https://github.com/danpejobo/geovita_processing_plugin/actions/workflows/test_plugin.yml)
[![GitHub Release](https://img.shields.io/github/v/release/danpejobo/geovita_processing_plugin)](https://github.com/danpejobo/geovita_processing_plugin/releases)

A QGIS plugin for different Geovita custom processing algorithms. This plugin is currently under development, so new features may be expected!
The Geovita processing provider for QGIS!

**New algorithms are added to this repo as they are developed, and new releases published!**

Status and Limitations
=====

- Implemented [REMEDY GIS RiskTool](https://github.com/norwegian-geotechnical-institute/REMEDY_GIS_RiskTool) to run from QGIS processing
- It is important to read the manual for this project. Especially if you want to perform `vulnerability analysis`. This will require you to edit the polygon layers containing the `building` features (see specifications section below).
- Supported filetypes: `.shp`
- If you have an "in memory" layer or other fileformats you will need to save it to a `.shp` file. This is a restriction imposed by the underlaying submodule.
- Projection of layers:
- All layers `need the same projection` before you can run the algorithms
- The algorithm does not (yet) reproject layers. This means you have to do it manually first.

Tools
=====
- **REMEDY GIS RiskTool** - These algorithms create a log directory in this location `%user%/Downloads/REMEDY`. For the moment this is hardcoded.
- `Begrens Skade - Excavation` The Begrens Skade - Excavation algorithm provides a comprehensive analysis of building settlements and risks associated with subsidence and inclination.
- `Begrens Skade - ImpactMap` The BegrensSkade ImpactMap alorithm calculates both short-term and long-term settlements that occur due to the establishment of a construction pit (this alg. takes a bit of time to run, open the log and refresh it to see the logged progress).
- `Begrens Skade - Tunnel` The BegrensSkade Tunnel alorithm provides a comprehensive analysis of building settlements and risks associated with subsidence and inclination due to tunnel excavation.
**New algorithms are added to the provider as they are developed**

QGIS Plugin
===========

This provider functions as a QGIS plugin (for QGIS >= 3.4) and is available via the standard QGIS plugins repository, so you can install it directly from within QGIS itself.
This provider functions as a QGIS plugin (for QGIS >= 3.28) and is available via the standard QGIS plugins repository, so you can install it directly from within QGIS itself.

The plugin adds a new group to the Processing Toolbox for "Geovita", containing sub-groups with tools and algorithms to perform different tasks.

If you enconter bugs of any sort, PLEASE consider reporting them through [the bugtracker at GitHub](https://github.com/danpejobo/geovita_processing_plugin/issues). Everyone benefits!

Specifications
==============

It is crucial to understand the limitations of the [REMEDY GIS RiskTool](https://github.com/norwegian-geotechnical-institute/REMEDY_GIS_RiskTool) calculation methods. I would highly suggest reading the paper and the manual within the REMEDY repository.
Check it out [here!](/geovita_processing_plugin/)

If you want to enable `vulnerability analysis` you will need some information on the buildings near the excavation/tunnel. You will need to add text fields to each feature in the building polygon layer for `Foundation`, `Structure` and `Condition`. See below for the allowed values!
- Class A is the best and D the worst.
- If i cell contains `-` it only means that this field only have 2 possible values.
- Copy/paste the values so they match!

| Class | Foundation | Structure | Condition |
|-------|---------------------------------------------------------|----------------------------------|------------------------|
| A | To bedrock | Steel | Excellent |
| A | Peler | A - Stål | A - Meget god tilstand |
| A | A - Til berg - Direktefundamentert, peler | - | - |
| B | Raft | Reinforced concrete | Good |
| B | Betong | B - Armert betong | B - God tilstand |
| B | B - På løsmasser - Hel plate (betong, såle) | - | - |
| C | Strip | Mixed | Medium |
| C | Grunnmur | C - Tre eller varierende | C - Brukbar tilstand |
| C | C - På løsmasser - Stripefundament (heller) | - | - |
| D | Wooden piles | Masonry | Bad |
| D | Trepeler | D - Murstein eller spesiell type | D - Dårlig |
| D | D - På løsmasser - Punkt- og trefundamenter (banketter) | - | - |
Overview
========
- Implemented [REMEDY GIS RiskTool](https://github.com/norwegian-geotechnical-institute/REMEDY_GIS_RiskTool) to run from QGIS processing

## Results
## Example results from the REMEDY GIS RiskTool
The following images show some example results. Both the excavation and the tunnel algorithm produces results for short and/or longterm settlements, but uses different calculation methods. The impact map calculates and illustrate total settlements in the impaced soil around the excavation.

| Loaded Layers | Short term | Long term | Impact map |
|---------------|------------|-----------|------------|
| ![Loaded layers](resources/example-short-term-layers.png) <br><br> The symbology of the loaded layers | ![Short term](resources/example-short-term.png) <br><br> Blue hatch is the excavation. Status of corners, walls and buildings | ![Long term](resources/example-long-short-term.png) <br><br> Blue hatch is the excavation. Dark background is the depth to bedrock raster used for long term settlements. | ![Impact map](resources/example-impact-map.png) <br><br> Red hatch is the excavation. The impact map for total settlements around the excavation. |


1 change: 0 additions & 1 deletion REMEDY_GIS_RiskTool
Submodule REMEDY_GIS_RiskTool deleted from df37af
6 changes: 6 additions & 0 deletions REQUIREMENTS_TESTING.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# For tests execution:
deepdiff
mock
flake8
pep257
pylint
Loading