Skip to content

Intel Extractor v0.2.3
Compare
Choose a tag to compare
@S4M8 S4M8 released this 02 Jul 05:05
· 4 commits to main since this release
v0.2.3
ee7a54c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Intel Extractor: v0.2.3 - Major Scraping Improvements and Database Integration

0.2.3 Release Notes:

  • Efficient Scraping: The scraping strategy has been revamped to directly target the Star Citizen API, eliminating the need to scroll through the DOM. This translates to faster and more reliable data extraction.
  • Organization Verification: The tool now includes functions to verify the existence of an organization before proceeding with data collection.
  • Pagination Handling: Intel Extractor can now determine the number of pages required to scrape all member dossiers within an organization, ensuring comprehensive data capture.
  • Detailed Citizen Data: Extracted information now includes not just basic details like name and handle, but also location and fluency information from individual member profiles (optional).
  • SQLite Database Integration: Scraped data is now stored in a centralized SQLite database, facilitating easier management and analysis.
  • Separate CSV Export: While data is stored in the database, a separate CSV file containing all scanned member URLs is still generated for reference.

What's Changed

  • Electron wrapper by @neon-nomad in #1
  • org-scan by @neon-nomad in #2
  • Add Existing Roster Input by @neon-nomad in #3
  • Scraping improvements by @neon-nomad in #4

Full Changelog: https://github.com/neon-nomad/intel-extractor/commits/v0.2.3

Contributors

S4M8
Assets 3
Loading