README.md

🛤️ kalauz

📥 Installation

1. Clone the repository
2. Run the following command in the terminal:

cd kalauz && make install

📦 Dependencies

• Python 3.10+
• MySQL 5.6+
  • Store your database password in a DATABASE_PASSWORD variable in a .env file in the root of the repository.
• A ConvertAPI account (pricing)
  • Store your API secret in a CONVERTAPI_SECRET variable in a file named .env in the root of the repository.

📖 About

A program for...

• processing rail network operator companies' internal, dirty spreadsheets of speed restrictions
• storing that data in a MySQL database
• visualizing the speed restrictions on a map.

📋 Features

• Processes the following companies' spreadsheets of SRs in the following formats:
  • MÁV Magyar Államvasutak Zrt. (.pdf converted to .xlsx)
  • GYSEV Zrt. (.xlsx)
• Stores them in a MySQL database with the following schema:
• Visualizes them on a map using pydeck.

⚙️ How it works

• Importing
  • The program reads the spreadsheets from the data/01_received directory.
  • It converts the .pdf files to .xlsx files using the ConvertAPI.
  • It stores the converted files in the data/02_converted directory.
  • It reads the .xlsx files and extracts the SRs.
• Processing
  • It categorizes the cause of the SRs using AI (scikit-learn):
    • feature extraction (student): TfidfVectorizer (term frequency–inverse document frequency vectorizer)
    • linear model (estimator / predictor – it can constantly learn): SGDClassifier (stochastic gradient descent classifier)
      • extending it to a multioutput estimator: MultiOutputClassifier
• Storing
  • It stores the SRs in a MySQL database.
• Visualizing
  • It visualizes the SRs on a map using pydeck.

📚 Reference

abbreviation	meaning
SR	speed restriction
TSR	temporary speed restriction
ASR	all speed restrictions
MU	multiple unit
MP	multipolygon

🏦 Funding

This project was a part of the scholarship application ÚNKP-23-1-I-BME-354 which was granted 1,000,000 HUF (~2,600 EUR in 2023) from the National Research, Development and Innovation Fund by the New National Excellence Program of the Hungarian Ministry for Culture and Innovation.

📜 License

This project is licensed under the GNU General Public License v3.0. See the license file (or the GPL-3.0 license tab on GitHub) for its full text.