Data Collector

Purpose

This tool does two things:

1. download all invasive species data list from Information of Korean Alien Species website.

2. parse them into a fully-structured, granularly filtered format that fits well for both JSON and SQL.

How to use

First, clone this repository and cd into it.

$ git clone https://github.com/eco3s/data-collector
$ cd data-collector

make sure that you have cargo and rustc (follow this guide) then run the following command to build project.

$ cargo build --release

Or you can run it directly

$ cargo run --release

above command will automatically build the project if necessary, then run compiled downloader, which will download all raw json species data in downloads folder.

Contributing

Before Commit

1. install pre-commit on your machine.

   If not, try following

   $ pip install pre-commit

   To ensure pre-commit is successfully installed, try following

   $ pre-commit -V

   Which will show installed version.

   Then, setup rest things by below instructions

   $ pre-commit autoupdate
   $ pre-commit install

   Then try running the pre-commit hooks by following command

   $ pre-commit run -a

   if these hooks run successfully, you are able to commit. But the above will show an error, keep reading.

2. make sure that you have installed rustup and configured channel as nightly.

   we must need nightly rustfmt toolchain to support unstable format options listed in rustfmt.toml.

   if not, try following

   $ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
   $ rustup default nightly

   To ensure rustfmt is successfully installed, try following

   rustfmt -V

   which will show that installed version is nightly.

License