Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loss persistence colombian forests #104

Open
wants to merge 40 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
40b417c
test
vicjulrin Mar 8, 2023
abbec9b
Add files via upload
vicjulrin Mar 10, 2023
2533e57
Set Nginx parameters to request size an CORS
ManuelStardust Mar 13, 2023
e8eb2ab
Set input polygon as parameter and output response as json
ManuelStardust Mar 13, 2023
ae739d2
Add files via upload
vicjulrin Mar 13, 2023
0bdd152
Add files via upload
vicjulrin Mar 13, 2023
1d35c94
Add files via upload
vicjulrin Mar 13, 2023
53660da
Delete scripts/1_IAvH directory
vicjulrin Mar 15, 2023
1c44c41
Delete scripts/IAvH_inputs directory
vicjulrin Mar 15, 2023
6b2466b
Delete scripts/1_IAvH_functions directory
vicjulrin Mar 15, 2023
da7b58c
Add files via upload
vicjulrin Mar 15, 2023
897e71b
Delete scripts/IAvH_inputs/p&p_collection directory
vicjulrin Mar 15, 2023
a52cb21
Add files via upload
vicjulrin Mar 15, 2023
be57c3e
Merge branch 'Loss_Persistence_Colombian_forests' of https://github.c…
vicjulrin Mar 15, 2023
d75988d
ignore
vicjulrin Mar 15, 2023
69d074b
ignore
vicjulrin Mar 15, 2023
f6c53eb
Update 0s
vicjulrin Mar 15, 2023
2c26410
Update wkt lines
vicjulrin Mar 15, 2023
3659771
update json
vicjulrin Mar 16, 2023
a9de8ae
update wkt polygon
vicjulrin Mar 16, 2023
a500187
Merge branch 'Loss_Persistence_Colombian_forests' of github.com:PEM-H…
ManuelStardust Mar 16, 2023
eef87bf
Set JSon output
ManuelStardust Mar 18, 2023
9067a54
Update 22_03
vicjulrin Mar 22, 2023
14b4adc
Merge branch 'Loss_Persistence_Colombian_forests' of https://github.c…
vicjulrin Mar 22, 2023
7caa5f0
Update area_ha
vicjulrin Mar 22, 2023
5d13d62
Update png
vicjulrin Mar 28, 2023
c47c311
Update colors and resolution
vicjulrin Mar 29, 2023
6cd9c94
Update colors
vicjulrin Mar 30, 2023
96925f5
Update png output
vicjulrin Mar 30, 2023
536dcf1
Polygon Forest LP input and output adjusments
ManuelStardust Apr 11, 2023
2b30205
Add files via upload
vicjulrin Jul 7, 2023
b338b1d
Update lossPersistence
vicjulrin Jul 26, 2023
03a4388
add table pp
vicjulrin Jul 26, 2023
e149a2a
Update outputfolder bridge
vicjulrin Jul 26, 2023
2ff447d
Revision time desarticulaado
vicjulrin Jul 28, 2023
5d59171
correct pp poligonos pequeños, areas no intersectadas, y orden de tab…
vicjulrin Aug 12, 2023
097f0df
gdalUtilities update
vicjulrin Aug 16, 2023
c0400a9
correct dims studyarea
vicjulrin Aug 16, 2023
0c21d61
pp etiquetas _
vicjulrin Aug 16, 2023
7c900a8
correct areas
vicjulrin Aug 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,8 @@
.Rhistory
**/.cache
**/.config
scripts/1_IAvH_functions/01_pp.R
Copy link
Contributor

@jmlord jmlord Oct 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why adding these in gitignore? Should probably be removed from .gitignore.

scripts/1_IAvH_functions/01_pp.yml
scripts/IAvH_inputs/p&p_collection/metadata.json
scripts/IAvH_inputs/p&p_collection/metadata.csv
*.tif
2 changes: 2 additions & 0 deletions http-proxy/conf.d/ngnix.conf
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
server {

listen 80;
client_max_body_size 8M;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like a bit of background to know why this is needed. We are downloading large files from the server, but when are we uploading large files?

add_header Access-Control-Allow-Origin "localhost:3000";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain the situation why this is needed? A comment could be added in code to help.

If needed, conf.d-prod/nginx.conf also needs to be updated, or this will have no effect on the compiled image.

server_name boninabox.com;

location /script/ {
Expand Down
97 changes: 97 additions & 0 deletions pipelines/01_pp.R
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be removed. It's in the wrong folder, and looks like a duplicate of scripts/lossPersistance/pp.R
Same for the .yml

Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
### funci?n check_collections

# Cargar librerias
packages_list<-list("magrittr", "terra", "raster")
invisible(lapply(packages_list, library, character.only = TRUE))

# Organizar directorios
args <- commandArgs(trailingOnly=TRUE)
outputFolder <- args[1]

# Cargar archivos de entrada
input <- rjson::fromJSON(file=file.path(outputFolder, "input.json"))

dir_wkt<- input$dir_wkt_polygon
dir_colection<- input$dir_colection

epsg_polygon<- input$epsg_polygon
resolution<- input$resolution
folder_output<- input$folder_output

wkt_polygon<- readLines(dir_wkt)
layers <- list.files(dir_colection, "\\.tif$", recursive = TRUE, full.names = TRUE)
json_colleciton_file <- list.files(dir_colection, "\\.json$", recursive = TRUE, full.names = TRUE)
meadata_collecion_file <- list.files(dir_colection, "\\.csv$", recursive = TRUE, full.names = TRUE)
metadata<- read.csv(meadata_collecion_file)

# Especificar carpea donmde guardar resultados
folder_results<- paste0(dirname(dirname(dirname(outputFolder))), "/", folder_output)
dir.create(folder_results)


# Especificar info area de estudio
vector_polygon<- terra::vect(wkt_polygon, crs= sf::st_crs(epsg_polygon)$proj4string ) %>% terra::as.polygons()
crs_polygon<- terra::crs(vector_polygon)
box_polygon<- sf::st_bbox(vector_polygon)


# Alinear con colecci?on
stac_collection<- gdalcubes::create_image_collection(files= layers, format= json_colleciton_file)

# Cargar cubo
cube_collection<- gdalcubes::cube_view(srs = crs_polygon, extent = list(t0 = gdalcubes::extent(stac_collection)$t0, t1 = gdalcubes::extent(stac_collection)$t1,
left = box_polygon[1], right = box_polygon[3],
top = box_polygon[4], bottom = box_polygon[2]),
dx = resolution, dy = resolution, dt = "P1Y", aggregation = "first", resampling = "first", keep.asp= F)

cube <- gdalcubes::raster_cube(stac_collection, cube_collection)

# Cortar cubo por area de estudio
cube_mask<- gdalcubes::filter_geom(cube, geom= wkt_polygon, srs = crs_polygon )


# Convertir cubo a raster
cube_stars <- stars::st_as_stars(cube_mask) %>% terra::rast() %>% setNames(names(cube))

collection_rast<- lapply(cube_stars, function(x) { if(any( is.na(summary(raster::raster(x))) )){NULL}else{x} } ) %>%
{Filter(function(x) !is.null(x), .)} %>% {setNames(., unlist(sapply(., function(x) names(x))) )} %>% terra::rast()

# estimar metricas de area
data_sum<- terra::freq(collection_rast, usenames=T) %>% dplyr::mutate(area_m2= count*resolution) %>% dplyr::select(-count) %>%
dplyr::rename(collection= layer) %>% dplyr::mutate(layer= sapply(.$collection, function(x) stringr::str_split(x, "_B*time")[[1]][1]) ) %>%
list(metadata) %>% plyr::join_all() %>% dplyr::group_by(layer) %>% dplyr::mutate(porcentaje= area_m2/sum(area_m2) )

data_sum$periodo<- unlist(strsplit(data_sum$layer, "_") %>% sapply(function(x) paste(x[c(2:3)], collapse = "_")), recursive = T)

names(data_sum)<-c("collection", "value", "area", "layer", "key", "percentage", "period")



# partir y guardar raster acorde a los valores
setwd(folder_results)
saveraster<- lapply( seq(nrow(data_sum)), function(i) {

x<- data_sum[i, ]

raster_val<- collection_rast[x$layer]
raster_val[!raster_val %in% x$value]= NA
raster_val[raster_val %in% x$value]= 1

terra::writeRaster(raster_val, paste0(x$layer, "_class_",x$class, ".tif"), overwrite=T)

} )




# guardar tabla resumen
setwd(folder_results)
write.csv(data_sum,"data_summ.csv")

# Imprimir resultado en logs
print(data_sum)

## Imprimir resultado - Formao json
output <- list("folder_output" = folder_results)
jsonData <- rjson::toJSON(output, indent=2)
write(jsonData, file.path(outputFolder,"output.json"))
35 changes: 35 additions & 0 deletions pipelines/01_pp.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
script: 01_pp.R
description: "This sample script shows how it works."
external_link: https://github.com/GEO-BON/biab-2.0
inputs:
dir_wkt_polygon:
label: dir_wkt_polygon
description: dir_wkt_polygon
type: text
example: '/scripts/IAvH_inputs/wkt_polygon_test.txt'
epsg_polygon:
label: epsg_polygon
description: epsg_polygon
type: numeric
example: 3395
dir_colection:
label: dir_colection
description: dir_colection
type: text
example: '/scripts/IAvH_inputs/p&p_collection'
resolution:
label: resolution
description: value in meters
type: numeric
example: 1000
folder_output:
label: folder_output
description: folder_output
type: text
example: 'p_p_studyarea_1000m2'

outputs:
folder_output:
label: folder_output
description: folder_output
type: text
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
0.0002500000
0.0000000000
0.0000000000
-0.0002500000
-79.0074509690
12.4582381791
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
0.0002500000
0.0000000000
0.0000000000
-0.0002500000
-79.0074509690
12.4582381791
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
0.0002500000
0.0000000000
0.0000000000
-0.0002500000
-79.0074509690
12.4582381791
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
0.0002500000
0.0000000000
0.0000000000
-0.0002500000
-79.0074509690
12.4582381791
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
0.0002500000
0.0000000000
0.0000000000
-0.0002500000
-79.0074509690
12.4582381791
177 changes: 177 additions & 0 deletions scripts/lossPersistence/input/Colombia_pp_collection/catalog.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not able to run the example locally, since the tiff files are missing. Could you please put the tiff files online and link to them? I see that your script supports it.

Another suggestion could be to put an external link to the files and have people download them to the userdata folder.

Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
{
"id": "Colombia_pp",
"type": "Collection",
"description": "Collection format for Biotablero (IAvH, 2022)",
"pattern": ".+\\.tif",
"images": {
"pattern": ".*(Colombia_pp-.{4}_.{2}_.{2}-.*)\\.tif"
},
"datetime": {
"pattern": ".*Colombia_pp-(.{4}_.{2}_.{2})-.*\\.tif",
"format": "%Y_%m_%d"
},
"features": {
"Colombia_pp-2005_12_31-pp_2000_2005": {
"id": "Colombia_pp-2005_12_31-pp_2000_2005",
"pattern": ".+Colombia_pp-2005_12_31-pp_2000_2005+.*\\.tif",
"assets": {
"Colombia_pp-2005_12_31-pp_2000_2005": {
"href": "Colombia_pp-2005_12_31-pp_2000_2005.tif",
"classes": {
"period_layer": ["2000-2005", "2000-2005", "2000-2005"],
"col": ["#c65453", "#92ab58", "#c5b599"],
"value": [0, 1, 2],
"classes": ["Perdida", "Persistencia", "No Bosque"]
}
}
},
"bbox": [-180, -90, 180, 90],
"properties": {
"proj:epsg": null,
"resolution": [300, 300]
}
},
"Colombia_pp-2010_12_31-pp_2006_2010": {
"id": "Colombia_pp-2010_12_31-pp_2006_2010",
"pattern": ".+Colombia_pp-2010_12_31-pp_2006_2010+.*\\.tif",
"assets": {
"Colombia_pp-2010_12_31-pp_2006_2010": {
"href": "Colombia_pp-2010_12_31-pp_2006_2010.tif",
"classes": {
"period_layer": ["2006-2010", "2006-2010", "2006-2010"],
"col": ["#c65453", "#92ab58", "#c5b599"],
"value": [0, 1, 2],
"classes": ["Perdida", "Persistencia", "No Bosque"]
}
}
},
"bbox": [-180, -90, 180, 90],
"properties": {
"proj:epsg": null,
"resolution": [300, 300]
}
},
"Colombia_pp-2015_12_31-pp_2011_2015": {
"id": "Colombia_pp-2015_12_31-pp_2011_2015",
"pattern": ".+Colombia_pp-2015_12_31-pp_2011_2015+.*\\.tif",
"assets": {
"Colombia_pp-2015_12_31-pp_2011_2015": {
"href": "Colombia_pp-2015_12_31-pp_2011_2015.tif",
"classes": {
"period_layer": ["2011-2015", "2011-2015", "2011-2015"],
"col": ["#c65453", "#92ab58", "#c5b599"],
"value": [0, 1, 2],
"classes": ["Perdida", "Persistencia", "No Bosque"]
}
}
},
"bbox": [-180, -90, 180, 90],
"properties": {
"proj:epsg": null,
"resolution": [300, 300]
}
},
"Colombia_pp-2021_12_31-pp_2016_2021": {
"id": "Colombia_pp-2021_12_31-pp_2016_2021",
"pattern": ".+Colombia_pp-2021_12_31-pp_2016_2021+.*\\.tif",
"assets": {
"Colombia_pp-2021_12_31-pp_2016_2021": {
"href": "Colombia_pp-2021_12_31-pp_2016_2021.tif",
"classes": {
"period_layer": ["2016-2021", "2016-2021", "2016-2021"],
"col": ["#c65453", "#92ab58", "#c5b599"],
"value": [0, 1, 2],
"classes": ["Perdida", "Persistencia", "No Bosque"]
}
}
},
"bbox": [-180, -90, 180, 90],
"properties": {
"proj:epsg": null,
"resolution": [300, 300]
}
}
},
"bands": {
"Colombia_pp-2005_12_31-pp_2000_2005": {
"id": "Colombia_pp-2005_12_31-pp_2000_2005",
"pattern": ".+Colombia_pp-2005_12_31-pp_2000_2005+.*\\.tif",
"assets": {
"Colombia_pp-2005_12_31-pp_2000_2005": {
"href": "Colombia_pp-2005_12_31-pp_2000_2005.tif",
"classes": {
"period_layer": ["2000-2005", "2000-2005", "2000-2005"],
"col": ["#c65453", "#92ab58", "#c5b599"],
"value": [0, 1, 2],
"classes": ["Perdida", "Persistencia", "No Bosque"]
}
}
},
"bbox": [-180, -90, 180, 90],
"properties": {
"proj:epsg": null,
"resolution": [300, 300]
}
},
"Colombia_pp-2010_12_31-pp_2006_2010": {
"id": "Colombia_pp-2010_12_31-pp_2006_2010",
"pattern": ".+Colombia_pp-2010_12_31-pp_2006_2010+.*\\.tif",
"assets": {
"Colombia_pp-2010_12_31-pp_2006_2010": {
"href": "Colombia_pp-2010_12_31-pp_2006_2010.tif",
"classes": {
"period_layer": ["2006-2010", "2006-2010", "2006-2010"],
"col": ["#c65453", "#92ab58", "#c5b599"],
"value": [0, 1, 2],
"classes": ["Perdida", "Persistencia", "No Bosque"]
}
}
},
"bbox": [-180, -90, 180, 90],
"properties": {
"proj:epsg": null,
"resolution": [300, 300]
}
},
"Colombia_pp-2015_12_31-pp_2011_2015": {
"id": "Colombia_pp-2015_12_31-pp_2011_2015",
"pattern": ".+Colombia_pp-2015_12_31-pp_2011_2015+.*\\.tif",
"assets": {
"Colombia_pp-2015_12_31-pp_2011_2015": {
"href": "Colombia_pp-2015_12_31-pp_2011_2015.tif",
"classes": {
"period_layer": ["2011-2015", "2011-2015", "2011-2015"],
"col": ["#c65453", "#92ab58", "#c5b599"],
"value": [0, 1, 2],
"classes": ["Perdida", "Persistencia", "No Bosque"]
}
}
},
"bbox": [-180, -90, 180, 90],
"properties": {
"proj:epsg": null,
"resolution": [300, 300]
}
},
"Colombia_pp-2021_12_31-pp_2016_2021": {
"id": "Colombia_pp-2021_12_31-pp_2016_2021",
"pattern": ".+Colombia_pp-2021_12_31-pp_2016_2021+.*\\.tif",
"assets": {
"Colombia_pp-2021_12_31-pp_2016_2021": {
"href": "Colombia_pp-2021_12_31-pp_2016_2021.tif",
"classes": {
"period_layer": ["2016-2021", "2016-2021", "2016-2021"],
"col": ["#c65453", "#92ab58", "#c5b599"],
"value": [0, 1, 2],
"classes": ["Perdida", "Persistencia", "No Bosque"]
}
}
},
"bbox": [-180, -90, 180, 90],
"properties": {
"proj:epsg": null,
"resolution": [300, 300]
}
}
}
}
Loading