Code

Update spiders.txt#289

Code

Open

acidvertigo wants to merge 27 commits intoosCommerce:masterosCommerce/oscommerce2:masterfrom acidvertigo:patch-22acidvertigo/oscommerce2:patch-22Copy head branch name to clipboard

+84-43

Commits

Commits on Dec 9, 2014

• Update spiders.txt

  Show description for 4fde7d1
  acidvertigo

  committed
  4fde7d1

  Copy full SHA for 4fde7d1

• change 008 spider to voltron

  acidvertigo

  committed
  49ac9ae

  Copy full SHA for 49ac9ae

• Remove AnyApexBot as not exists anymore

  acidvertigo

  committed
  9200ef4

  Copy full SHA for 9200ef4

• Remove B-l-i-t-z-B-O-T as does not exists anymore

  acidvertigo

  committed
  c6e4b5d

  Copy full SHA for c6e4b5d

• Remove BillyBobBot as does not exists anymore

  acidvertigo

  committed
  8d040f6

  Copy full SHA for 8d040f6

• remoe Boithobot as does not works anymore

  acidvertigo

  committed
  a984faf

  Copy full SHA for a984faf

• Remove btbot as use google search results

  acidvertigo

  committed
  f0dda94

  Copy full SHA for f0dda94

• remove CatchBot as does not exists anymore

  acidvertigo

  committed
  8104436

  Copy full SHA for 8104436

• Remove Cerberian Drtrs as it is a filtering tool

  acidvertigo

  committed
  7bb8b73

  Copy full SHA for 7bb8b73

• Remove Charlotte as it use yahoo search results

  acidvertigo

  committed
  3c4f48b

  Copy full SHA for 3c4f48b

• Remove ConveraCrawler cosmos spiders as index not html pages

  acidvertigo

  committed
  ee739dc

  Copy full SHA for ee739dc

• Remove Covario IDS as does not seems to work anymore

  acidvertigo

  committed
  87c7dcb

  Copy full SHA for 87c7dcb

• Remove DiamondBot as seems to not exists anymore

  acidvertigo

  committed
  1a6046a

  Copy full SHA for 1a6046a

• Remove discobot as does not exists anymore

  acidvertigo

  committed
  5102b28

  Copy full SHA for 5102b28

• Remove EARTHCOM.info as does not exists anymore

  acidvertigo

  committed
  d75908a

  Copy full SHA for d75908a

• Remove EmeraldShield.com WebBot as does not exists anymore

  acidvertigo

  committed
  4832988

  Copy full SHA for 4832988

• remove EsperanzaBot as does not exists anymore

  acidvertigo

  committed
  10575a9

  Copy full SHA for 10575a9

• Remove FDSE robot as is an internal serch engine for sites

  acidvertigo

  committed
  2963a62

  Copy full SHA for 2963a62

• Remove FindLinks as actually is not working

  acidvertigo

  committed
  9dc86a0

  Copy full SHA for 9dc86a0

• Remove g2crawler as it is not used anymore

  acidvertigo

  committed
  e753ed5

  Copy full SHA for e753ed5

• remove Gaisbot as does not exists anymore

  acidvertigo

  committed
  08b4fee

  Copy full SHA for 08b4fee

• Remove genieBot as does not exists anymore

  acidvertigo

  committed
  8a9ee7e

  Copy full SHA for 8a9ee7e

• Remove GurujiBot as does not exists anymore

  acidvertigo

  committed
  c1c02a0

  Copy full SHA for c1c02a0

• remove HappyFunBot as does not exists anymore

  acidvertigo

  committed
  bcf88ce

  Copy full SHA for bcf88ce

• Remove small and not existent spiders user agents

  acidvertigo

  committed
  b6b6e07

  Copy full SHA for b6b6e07

• Convert all to lowercase

  acidvertigo

  committed
  a958130

  Copy full SHA for a958130

• Fix typo

  acidvertigo

  committed
  a8097b4

  Copy full SHA for a8097b4