Skip to content

AIL Framework version 3.5 released with a new flexible crawler manager, built-in Passive DNS sensor and many fixes

Compare
Choose a tag to compare
@adulau adulau released this 31 Mar 13:24
· 967 commits to master since this release
c76986d

AIL Framework version 3.5 released with a new flexible crawler manager, built-in Passive DNS sensor and many fixes.

ail-email-corr
ail-key-leak
ail-crawler-manager

Changes

  • [passivedns] D4server port. [Terrtia]
  • [v3.5] install pyAIL. [Terrtia]
  • [passivedns] D4server port. [Terrtia]
  • [passiveDns] add dns records. [Terrtia]
  • [passiveDns D4 Client] add passiveDns D4 Client. [Terrtia]
  • [crawler] add test + relaunch crawlers + major fixs. [Terrtia]
  • [update] add v3.5 update. [Terrtia]
  • [doc] Splash Manager Configuration. [Terrtia]
  • [UI] add update note. [Terrtia]
  • [IPAddress] Add Debug + Check if option is empty. [Terrtia]
  • [splash manager] update enpoints + use Splash name to restart docker.
    [Terrtia]
  • [merge master] [Terrtia]
  • [domains search] search domains by name. [Terrtia]
  • [screenshot + har directory] add option to change screenshots
    directory. [Terrtia]
  • [crawler_manager] UI edit config + fix crawler queues. [Terrtia]
  • [config] save config on DB + default values. [Terrtia]
  • [crawlers manager] show setings. [Terrtia]
  • [crawler] show all crawlers type on dashboard. [Terrtia]
  • [crawler] manage crawlers. [Terrtia]
  • [crawler] crawler queue + restart docker on error. [Terrtia]
  • [core module] disable phone module by default. [Terrtia]
  • Merge master -> crawler_manager. [Terrtia]
  • [crawler manager] get all splash dockers, proxies and launch all
    crawlers. [Terrtia]

Fixes

  • [crawler] crawler test: remove print. [Terrtia]
  • [crawler UI] remove unused crawlers configuration. [Terrtia]
  • [crawler] user agent + splash restart. [Terrtia]
  • [crawler UI] crawler by domain type, remove old dashboard. [Terrtia]
  • [MISP Export UI] object type selector. [Terrtia]
  • [UI] fix navbar: invalid url. [Terrtia]
  • [crawler] typo. [Terrtia]
  • [crawler] typo. [Terrtia]
  • [crawler] typo. [Terrtia]
  • [Crawler] faup. [Terrtia]
  • [Launcher] launch core module: Crawler_manager. [Terrtia]
  • [cralers] remove debug. [Terrtia]
  • [Domain search] fix regex. [Terrtia]
  • [API ACL] avoid user_no_api users to access the API. [Terrtia]
  • [Module Queue] module without subscriber. [Terrtia]
  • [term Tracker] TimeoutException. [Terrtia]
  • [API ACL] read_only user role. [Terrtia]
  • [OVERVIEW] [Terrtia]
  • [gitignore] [Terrtia]
  • [showDomain] empty father field. [Terrtia]
  • [showDomain] empty father field. [Terrtia]
  • [redis cache] remove old Redis_Data_Merging db. [Terrtia]
  • [redis cache] remove old paste_name db. [Terrtia]
  • [background updater] Don't launch updates on fresh install. [Terrtia]
  • [v3.4 update] fix update progress. [Terrtia]
  • [background updater] remove completed updates. [Terrtia]
  • [install] force virtual environmemt activation. [Terrtia]

Other

  • Merge branch 'master' into crawler_manager. [Terrtia]

  • Merge pull request #89 from ail-project/crawler_manager. [Alexandre
    Dulaunoy]

    Crawler manager

  • Merge branch 'master' into crawler_manager. [Terrtia]

  • Merge branch 'master' into crawler_manager. [Terrtia]

  • Merge pull request #536 from simonsigre/patch-1. [Alexandre Dulaunoy]

    Typo in placeholder "Optionnal" --> "Optional"

  • Typo in placeholder "Optionnal" --> "Optional" [Simon Sigré]

    Updated placeholder value to correct typo;
    "Optionnal" --> "Optional"

  • Merge branch 'master' of github.com:CIRCL/AIL-framework. [Alexandre
    Dulaunoy]

  • Merge pull request #534 from simonsigre/patch-1. [Alexandre Dulaunoy]

    Added 'wget' as a dependency

  • Added 'wget' as a dependency. [Simon Sigré]

    'wget' is a dependency as without this additional components won't download as part of initial install, one such example is;

    wget -O /ail-framework/ardb/src/../deps/jemalloc-5.1.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/5.1.0/jemalloc-5.1.0.tar.bz2 && \
    
  • Merge branch 'master' into crawler_manager. [Terrtia]

  • Merge branch 'master' into crawler_manager. [Terrtia]

  • Merge branch 'master' into crawler_manager. [Terrtia]