Skip to content

Commit

Permalink
early version
Browse files Browse the repository at this point in the history
  • Loading branch information
hammerdirt committed Sep 25, 2024
1 parent 24f1691 commit 7f2804d
Show file tree
Hide file tree
Showing 217 changed files with 27,338 additions and 4,645 deletions.
Binary file added _build/.doctrees/bern_city_example.doctree
Binary file not shown.
Binary file added _build/.doctrees/bern_example.doctree
Binary file not shown.
Binary file added _build/.doctrees/bern_lakes_example.doctree
Binary file not shown.
Binary file added _build/.doctrees/biel_city_example.doctree
Binary file not shown.
Binary file added _build/.doctrees/bielersee_example.doctree
Binary file not shown.
Binary file added _build/.doctrees/environment.pickle
Binary file not shown.
Binary file added _build/.doctrees/extracting_topo_data.doctree
Binary file not shown.
Binary file added _build/.doctrees/project.doctree
Binary file not shown.
Binary file added _build/.doctrees/references.doctree
Binary file not shown.
Binary file added _build/.doctrees/the_llm_assistant.doctree
Binary file not shown.
Binary file added _build/.doctrees/titlepage.doctree
Binary file not shown.
4 changes: 4 additions & 0 deletions _build/html/.buildinfo
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 041481c9bbd721076049b1dc858ca2f1
tags: 645f666f9bcd5a90fca523b33c5a78b7
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added _build/html/_images/hammerdirt.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added _build/html/_images/situation_map.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added _build/html/_images/situation_map1.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added _build/html/_images/situation_map2.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added _build/html/_images/situation_map3.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added _build/html/_images/situation_map4.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
840 changes: 840 additions & 0 deletions _build/html/_sources/bern_city_example.md

Large diffs are not rendered by default.

933 changes: 933 additions & 0 deletions _build/html/_sources/bern_example.md

Large diffs are not rendered by default.

901 changes: 901 additions & 0 deletions _build/html/_sources/bern_lakes_example.md

Large diffs are not rendered by default.

824 changes: 824 additions & 0 deletions _build/html/_sources/biel_city_example.md

Large diffs are not rendered by default.

951 changes: 951 additions & 0 deletions _build/html/_sources/bielersee_example.md

Large diffs are not rendered by default.

884 changes: 884 additions & 0 deletions _build/html/_sources/extracting_topo_data.ipynb

Large diffs are not rendered by default.

112 changes: 112 additions & 0 deletions _build/html/_sources/project.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Proposal

The challenge in front of us is to transform the observations into accurate and actionable information for two different purposes. There is the
administration of the territory and identification of priorities with regard to plastics in the environment. Then, once the priorities have been established these
new indicators must be reported on. This project concerns the infrastructure and processes necessary to identify priorities and develop indicators to reduce plastics
in the environment from litter density counts.

Therefore, we propose a harmonised reporting system that is administered by regional or national NGOS in collaboration with an academic partner with
the stated goal of producing reliable reports for the public administration and others who communicate on such matters. This allows administrations to
identify priorities and set baseline values according to agreed upon and observed indicators.

The first report proved that a relatively small team could cover a large territory with the correct technology. This remains the case.
That is why we see this project, in its current iteration, specifically attractive to stakeholders with an internal data-science team.
Stakeholders with experience in python or R will feel right at home in the development environment and will be the source of many improvements.
For those interested in learning the basics, we are here to coach them and help them in their projects.

The proposal is thus to establish a hub that is dedicated to the collection, cleaning, aggregating and distribution of litter density data.
The distribution of the data has the primary goal of producing decision support products to local administrations and other organizations charged with
reducing plastics in the environment. The _hub_ is staffed by three people and who conduct surveys, produce value added research, teach other
NGOs to collect samples, and respond to requests from local administrations. Activities or changes in protocol are considered in consultation with academic advising team.

## Goals

> * Develop a reporting and research tool for stakeholders in the waste management, life cycle assessment and environmental monitoring sectors.
> * Increase capacity of stakeholders to monitor and report observations according to the current standard in the federal report using open source tools.
> * Assist in the reduction and prevention of plastics in the environment
:::{dropdown} Deliverables: four year contract
1. 800 samples
2. Revised updated federal report
3. New cantonal and municipal reports
4. Cloud service for cantonal and municipal reporting
5. Chat service for endusers to develop indi
:::

:::{dropdown} Objectives
1. Collect 400 - 500 samples in a 12 month period
* reduce the number of items reported as _fragmented_ or _Gfrags_
1. This value can range from 20% - 30% of the total items collected
* include polymer identification for a subset of samples
1. Subset should support an academic publication
2. Subset should support a regional assessment of polymer types
* Increase knowledge sharing between surveyors
* Capacity building for surveyors

2. Increase the number of associations competent to collect and report data
* Develop one association on Lac de Constance or Zurichsee
* Develop training materials and protocols with ASL

3. Structure report specifically for cantonal and municipal reporting
* Revise and improve the current report
* Consider the format of the report given cantonal requirements (attached)

4. Build upon previous research and reports
* Continue to develop modeling and parameter estimation for litter density
* Focus on topographical features and densities
* Consider polymer types and densities

5. Define groups of objects of interest with stakeholders
* single use plastics
* agricultural plastics
* construction plastics
* industrial plastics

6. Develop a cloud first infrastructure and chat agent
* Move all data and software assets to Infrastructure as a Service (IaaS)
* Develop machine learning and artificial intelligence pipelines for data analysis
* Develop Automated programming interfaces (APIs) for data collection and reporting
* Develop a user interface for data collection and reporting
* Deploy chat agent
:::

:::{dropdown} Benchmarks
1. Year one: Proposed updates and revisions to the federal report. Proposed model for cantonal and municipal reporting.
* Formal research proposal for polymer identification
* Formal research proposal for topographical features
* Functioning prototype of cloud service
* 100 samples collected
* stakeholder meeting
2. Year two:
* Assessment of software and data collection
* Revised report with current data
* Update on capacity building
* End of year two:
1. Presentation of revised report with current data
2. Summary of capacity building
3. 400 samples
3. stakeholder meeting
3. Year three:
* Presentation of feedback
* stakeholder meeting
* Proposed changes to app and report
* Final report:
1. implementation of feedback
2. 200 samples
3. cloud service for cantonal and municipal reporting
4. Year four:
* Testing and implementation
* 100 samples

:::

:::{dropdown} Cost

> Less than 1% of the estimated cost to prevent and remove litter annualy in Switzerland. [BAFU/OFEV - litter costs](https://www.bafu.admin.ch/bafu/en/home/topics/waste/publications-studies/publications/litter-dropping-costs-money.html)
:::


## Interested ?

Interested administrations or other potential financial partners should contact roger at hammerdirt.
74 changes: 74 additions & 0 deletions _build/html/_sources/references.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# References

## Collection protocols and previous studies in Switzerland

(mlw_2013)=
__Guidance on Monitoring of Marine Litter in European Seas__, Galgani et all, EUR 26113. Luxembourg (Luxembourg):
Publications Office of the European Union; 2013. JRC83985 [https://publications.jrc.ec.europa.eu/repository/handle/JRC83985](https://publications.jrc.ec.europa.eu/repository/handle/JRC83985)

(mlw_2023)=
__Guidance on the Monitoring of Marine Litter in European Seas An update to improve the harmonised monitoring of
marine litter under the Marine Strategy Framework Directive__, Galgani et all., EUR 31539 EN, Publications Office of the European Union, Luxembourg, 2023,
ISBN 978-92- 68-04093-5, doi:10.2760/59137, JRC133594 [https://literatur.thuenen.de/digbib_extern/dn067364.pdf](https://literatur.thuenen.de/digbib_extern/dn067364.pdf)

(eu_threshold)=
__A European threshold value and assessment method for macro litter on coastlines__, Publications Office of the
European Union, Van Loon, et all , [https://publications.jrc.ec.europa.eu/repository/handle/JRC121707](https://publications.jrc.ec.europa.eu/repository/handle/JRC121707), ISBN 978-92-76-21444-1, doi:10.2760/54369, JRC121707.

(basin_organization)=
__Transfer of waste and plastics in aquatic environments__, Christophe Brachet, International Netwwork of Basin Organizations, 2024, [INBO](https://www.inbo-news.org/)

(land_use_role)=
__Revealing the role of land-use features on macrolitter distribution in Swiss freshwaters__, Schreyers et al., 2022, Environmental Pollution Journal, [https://doi.org/10.1016/j.envpol.2024.124911](https://doi.org/10.1016/j.envpol.2024.124911) article: [sciencedirect](https://www.sciencedirect.com/science/article/pii/S0269749124016257)

(ofev_dechets_cours_deau)=
__Des déchets aux abords des lacs et cours d’eau suisses__, Office fédéral de l'environment, 2021, [OFEV](https://www.bafu.admin.ch/bafu/fr/home/themes/dechets/dossiers/des-dechets-aux-abords-des-lacs-et-cours-d-eau-suisses.html)

(plastock)=
__Plastock: Estimation of plastics on the shores of Lake Geneva, switzerland__, _Pochelon et al._, 2023, [Plastock](https://associationsauvegardeleman.github.io/plastock/)

(OFEV)=
__Identification, quantification and analysis of observable anthropogenic litter along Swiss lake systems__, Erismann et al., OFEV. 2021. [IQAASL](https://hammerdirt-analyst.github.io/IQAASL-End-0f-Sampling-2021/index.html)

(SLR)=
__Swiss Litter Report__, Gabrielle, Kuhl; Pascal, Blahrer; 2018; [Stopp research](http://stoppp.org/)

## Math and geospatial

(scipy)=
__scipy: Fundamental Algorithms for Scientific Computing in Python__, _Virtanen et al_, Nature Methods, 2020, doi: 10.1038/s41592-019-0686-2, [scipy](https://www.scipy.org/).

(pandas)=
__pandas__, _The pandas development team_, Python software, 2020, doi: 10.5281/zenodo.3509134, url: [https://doi.org/10.5281/zenodo.3509134](https://doi.org/10.5281/zenodo.3509134).

(python_3)=
__Python 3 Reference Manual__, _Guido Van Rossum and Fred L. Drake_, CreateSpace, 2009, isbn: 1441412697, [python](https://www.python.org/).

(geopandas)=
__geopandas__, _Joris Van den Bossche et al_, Python software, 2020, doi: 10.5281/zenodo.3509134, url: [https://doi.org/10.5281/zenodo.3509134](https://doi.org/10.5281/zenodo.3509134).

(swisstlm)=
__swissTLMRegio: Le modèle numérique de la Suisse à petite échelle__, Swiss Federal office of topography, [swiss topo](https://www.swisstopo.admin.ch/fr/modele-du-territoire-swisstlmregio)

(myst)=
__MyST: Markedly Structured Text__, _Executable Book Project_, 2021, [https://myst-parser.readthedocs.io/en/latest/](https://myst-parser.readthedocs.io/en/latest/)

(scikit-learn)=
__scikit-learn: Machine Learning in python__, _Pedregosa et al_, Journal of Machine Learning Research, 2011, doi: 10.1109/MCSE.2007.55, [scikit-learn](https://scikit-learn.org/stable/).

(matplotlib)=
__Matplotlib: A 2D graphics environment__, _Hunter, J. D._, Computing in Science & Engineering, 2007, doi: 10.1109/MCSE.2007.55, [matplotlib](https://matplotlib.org/).

(bayes_gelman)=
__Bayesian Data Analysis - Third edition__, _Andrew Gelman_, Columbia university, 2021, [http://www.stat.columbia.edu/~gelman/book/BDA3.pdf](http://www.stat.columbia.edu/~gelman/book/BDA3.pdf), doi: 10.1201/b16018.

(think_bayes)=
__Think Bayes 2ed__, _Alan Downey_, Green Tea Press, 2021, [http://allendowney.github.io/ThinkBayes2/](http://allendowney.github.io/ThinkBayes2/)

(bayes_hackers)=
__Bayesian Methods for Hackers__, _Cameron Davidson-Pilon_, Addison-Wesley Professional, 2021, [https://github.
com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers](https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers),
doi: 10.5555/2851115

(bayes_jaynes)=
__Probability Theory: The logic of science__, _Edwin Jaynes_, Washington University, 2003, isbn: 0521592712. [the logic of science](https://bayes.wustl.edu/etj/prob/book.pdf)
58 changes: 58 additions & 0 deletions _build/html/_sources/the_llm_assistant.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# The role of the llm assistant

__Definition of the situation, problem and solution__

## Reporting volunteer observations

Alot of ink has been spilled on the topic of citizen science and the role of volunteers in data collection. We assume that a
a reasonable person would place value on the collective observations of a segment of the population in reference to a particular
topic. We also assume that the value of any such collective observations is largely dependent on the method in which the
observations were made and the transparency of the analytical process.

Beach litter data has two broad purposes: 1. To inform policy and 2. To inform the public. However, an informed policy at
the national level is not the same at the regional level. Resource allocation and policy implementation are at the
cantonal and municipal level. Thus, reasonable people at various levels of the administrative hierarchy may not have access to
the information needed to make informed decisions.

## Beach litter surveys

The European Environment Agency (EEA) has been working with volunteers to collect data on marine litter. The volunteers
have been collecting data on the amount and type of litter found on beaches in Europe. The collection method is defined
in the _Guide to monitoring marine litter in European seas_. Based on the analysis of the collected data a suggested
threshold value, and a method to compute it was introduced in 2020. The threshold value is based on the principal of
precaution regarding exposure to plastics. In switzerland monitoring of shoreline litter along lakes and rivers
has followed the same protocol with minor modifications due to topography since 2015.

## Data availability and reporting

The availability of the data is largely dependent on local NGOs and volunteers. There is no legal requirement to report litter data.
The federal report, modeled after the EEA report, provides a format and a method for aggregating and reporting but no
infrastructure to collect and report the data. This infrastructure is non-existent at the cantonal level. In the Léman region
data collection and reporting has standardized because of the work of local NGOs. The most recent report was published in 2024.
This report was not aggregated at the cantonal or municipal level either.

## Environmental monitoring and protection

Environmental monitoring and protection are the responsibility of the cantons in Switzerland. The cantons can and do delegate
certain responsibilities to the municipalities. Thus, a report at the national level provides an overview of the situation
but provides very little value in the way enabling action at the cantonal or municipal level. Even though the data is collected at the
municipal level it is not aggregated and reported at the cantonal level.

## Cantonal and municipal resources

Each canton and municipality has a section or department responsible for environmental pollution. These departments, for the most part,
are already working with limited resources. Managers of these departments may be more than competent in interpreting a report but lack the
additional human resources necessary to collect and prepare reports for their own canton or municipality. The cause of this is two fold: 1.
The lack of common infrastructure for data collection and reporting and 2. Inexperience in domain specific data handling and analysis

## The solution

We provide a solution that maximizes the efficiency of the reporting process by leveraging large language models (llms) and
standardized scientific computing. The llm assistant produces a rough draft of the standardized report, ready to be reviewed and
edited by the domain expert (or the responsible person). This reduces the cost to report the data and keeps a human in
the analytical loop. More of the resources can be allocated to collection and communication of the data. Or if you look at this way,
small investments in data collection will lead to large returns in communication at the municipal and cantonal level.

Ultimately, the goal is that those who must act to reduce pollution have the data in the format that they need to make informed
decisions. The llm assistant uses the current monitoring methods and the capabilities of the llms and scientific
computing to provide a solution that is both efficient, transparent and appropriate to all scales of governance and stakeholders.
70 changes: 70 additions & 0 deletions _build/html/_sources/titlepage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Reporting volunteer observations

This is the proof of concept for a distributed reporting system of marine litter density using data from volunteers.

:::{dropdown} Recent changes

__September 25:__ Added llm assisted reporting

__June 24:__ Added responses to use cases

__June 22:__ Added methods to calculate landuse profile using Geopandas.


:::

::::{grid}

:::{grid-item}

This is an application of large language models (llms) and machine learning to provide summary reports of volunteer observations
of marine litter using the OSPAR, JRC or NOAA methods for counting beach litter.

The application provides summary reports of exising data and provides forecasts for a region, city or body of water. The llm
acts first as a transcriber of test results and then as an assistant to provide a written report and resource package that can
be either used directly or integrated into other documents.

:::

:::{grid-item-card}

```{image} resources/hammerdirt.png
:alt: bob
:class: bg-primary mb-1
:width: 400px
:align: center
```
:::
::::
::::{grid}

:::{grid-item}


This application minimizes the cost of reporting and evaluating beach litter density and uses geospatial analysis to leverage
the use of data from other disparate locations. Most importantly, decision makers and stakeholders have access to reliable,
readable reports that allow them to communicate effectively and make informed decisions.

The data in these reports is a combination of observations from variety of groups in Switerland since 2015. The observations were recorded using an interpretation of the _Guide for Monitoring Marine Litter on European Seas_ [The guide](https://mcc.jrc.ec.europa.eu/main/dev.py?N=41&O=439&titre_chap=TG%20Litter&titre_page=Guidance%20for%20the%20Monitoring%20of%20Marine%20Litter). The guide and the monitoring of beach litter are part of decades of research, here is the brief history [A Brief History of Marine Litter Research](https://link.springer.com/chapter/10.1007/978-3-319-16510-3_1).

:::
:::{grid-item}
__Associated projects__

* https://hammerdirt-analyst.github.io/IQAASL-End-0f-Sampling-2021/
* https://hammerdirt-analyst.github.io/solid-waste-team/titlepage.html
* https://hammerdirt-analyst.github.io/landuse/titlepage.html
* https://associationsauvegardeleman.github.io/plastock/
* https://hammerdirt-analyst.github.io/finding-one-object/titlepage.html


:::

:::{grid-item}
:columns: 12

__THE TEAM__

Shannon Erismann, Montserrat Filella, Téo Gursoy, Christian Ludwig, Roger Erismann, Bettina Siegenthaler
:::
::::
Loading

0 comments on commit 7f2804d

Please sign in to comment.