77 different robots | Hits | Bandwidth | Last visit |
Yahoo Slurp | 77322+807 | 103.79 GB | 31 Jul 2008 - 23:59 |
MSNBot | 54155+1542 | 6.88 GB | 31 Jul 2008 - 23:58 |
Unknown robot (identified by 'crawl') | 53859+1517 | 3.74 GB | 31 Jul 2008 - 23:49 |
Googlebot | 42414+95 | 50.77 GB | 31 Jul 2008 - 23:56 |
Nutch | 7983+123 | 14.03 GB | 31 Jul 2008 - 23:20 |
MSNBot-media | 6410+204 | 1.50 GB | 31 Jul 2008 - 23:21 |
GigaBot | 4914+174 | 1.18 GB | 31 Jul 2008 - 23:55 |
Ask | 4308+113 | 104.44 MB | 31 Jul 2008 - 12:14 |
NimbleCrawler | 3329+1 | 381.22 MB | 04 Jul 2008 - 00:16 |
BaiDuSpider | 2600+8 | 496.92 MB | 31 Jul 2008 - 23:06 |
twiceler | 1509+59 | 39.75 MB | 29 Jul 2008 - 06:07 |
Unknown robot (identified by 'bot*') | 1383+140 | 687.15 MB | 31 Jul 2008 - 22:50 |
Unknown robot (identified by empty user agent string) | 625+350 | 375.68 MB | 31 Jul 2008 - 22:29 |
Turn It In | 838+5 | 1.60 GB | 18 Jul 2008 - 21:54 |
Sogou Spider | 820+1 | 26.79 MB | 31 Jul 2008 - 22:27 |
Harvest | 763 | 269.79 MB | 28 Jul 2008 - 22:56 |
Speedy Spider | 400+76 | 1.47 GB | 31 Jul 2008 - 22:19 |
Unknown robot (identified by 'robot') | 343+81 | 24.36 MB | 31 Jul 2008 - 21:06 |
Voila | 330+61 | 235.27 MB | 28 Jul 2008 - 09:58 |
OutfoxBot/YodaoBot | 213+79 | 7.32 MB | 31 Jul 2008 - 22:21 |
Alexa (IA Archiver) | 253+31 | 244.01 MB | 31 Jul 2008 - 00:15 |
Perl tool | 274+1 | 16.05 MB | 30 Jul 2008 - 21:31 |
Unknown robot (identified by hit on 'robots.txt') | 0+258 | 26.23 KB | 31 Jul 2008 - 16:23 |
Unknown robot (identified by 'discovery') | 169+10 | 580.54 KB | 31 Jul 2008 - 08:43 |
Heritrix | 167+5 | 14.95 MB | 29 Jul 2008 - 19:08 |
WGet tools | 155+1 | 178.63 MB | 30 Jul 2008 - 18:57 |
EasyDL | 136 | 2.14 MB | 31 Jul 2008 - 09:44 |
Java (Often spam bot) | 119+14 | 66.54 MB | 31 Jul 2008 - 09:26 |
Exabot | 110+5 | 2.98 MB | 31 Jul 2008 - 21:24 |
Jakarta commons-httpclient | 66+5 | 4.41 MB | 31 Jul 2008 - 07:04 |
CFNetwork | 64 | 431.84 MB | 30 Jul 2008 - 17:39 |
DoCoMo | 62+2 | 3.04 MB | 31 Jul 2008 - 17:38 |
The web archive (IA Archiver) | 0+43 | 5.50 KB | 31 Jul 2008 - 20:24 |
Holmes | 30+10 | 613.53 KB | 30 Jul 2008 - 02:55 |
Grub.org | 24+16 | 6.50 MB | 22 Jul 2008 - 17:57 |
Python-urllib | 31 | 27.86 MB | 31 Jul 2008 - 07:10 |
BBot | 30 | 69.86 MB | 31 Jul 2008 - 01:44 |
MJ12bot | 23+6 | 217.64 KB | 27 Jul 2008 - 07:10 |
Geniebot | 14+8 | 47.89 MB | 19 Jul 2008 - 03:19 |
Unknown robot (identified by 'spider') | 13+5 | 10.87 MB | 29 Jul 2008 - 13:41 |
WebCollage | 17 | 9.55 MB | 23 Jul 2008 - 13:00 |
Custo | 16 | 38.63 MB | 29 Jul 2008 - 06:35 |
MSIECrawler | 5+10 | 9.11 KB | 31 Jul 2008 - 03:53 |
HTMLParser | 14 | 2.65 MB | 31 Jul 2008 - 16:28 |
larbin | 9+5 | 82.55 KB | 28 Jul 2008 - 17:02 |
BSpider | 6+7 | 3.55 MB | 30 Jul 2008 - 19:27 |
Unknown robot (identified by '*bot') | 6+6 | 2.39 MB | 18 Jul 2008 - 13:52 |
MSRBOT | 6+6 | 217.00 KB | 15 Jul 2008 - 07:40 |
yacy | 10+1 | 678.95 KB | 13 Jul 2008 - 01:48 |
boitho.com-dc | 9+2 | 1.22 MB | 28 Jul 2008 - 05:12 |
Web Core / Roots | 10 | 652.43 KB | 25 Jul 2008 - 17:10 |
Checkbot | 10 | 0 | 31 Jul 2008 - 03:50 |
UCSD Crawl | 5+4 | 191.67 MB | 29 Jul 2008 - 02:59 |
Yandex bot | 6+1 | 270 Bytes | 30 Jul 2008 - 18:35 |
NG 2.x (Exalead) | 0+6 | 786 Bytes | 28 Jul 2008 - 15:22 |
Ingrid | 5 | 695 Bytes | 14 Jul 2008 - 23:40 |
The World Wide Web Worm | 5 | 0 | 30 Jul 2008 - 04:58 |
Biglotron | 4 | 28.01 KB | 04 Jul 2008 - 10:55 |
HTTrack off-line browser | 3+1 | 13.53 MB | 28 Jul 2008 - 19:57 |
WordPress | 4 | 18.31 KB | 29 Jul 2008 - 17:14 |
Fish search | 4 | 78.22 KB | 30 Jul 2008 - 14:32 |
Girafabot | 0+4 | 524 Bytes | 18 Jul 2008 - 04:32 |
Gaisbot | 1+2 | 1012.23 KB | 28 Jul 2008 - 02:49 |
Unknown robot (identified by 'hunter') | 3 | 28.40 KB | 31 Jul 2008 - 11:33 |
Netluchs | 1+1 | 270 Bytes | 17 Jul 2008 - 10:55 |
Unknown robot (identified by 'checker') | 1+1 | 270 Bytes | 04 Jul 2008 - 23:02 |
lwp-request | 2 | 0 | 31 Jul 2008 - 19:23 |
ichiro | 1+1 | 270 Bytes | 28 Jul 2008 - 07:44 |
LinkChecker | 1+1 | 131 Bytes | 28 Jul 2008 - 12:21 |
lwp-trivial | 2 | 81.50 KB | 22 Jul 2008 - 17:40 |
FaceBook bot | 1 | 184.79 KB | 25 Jul 2008 - 04:30 |
SpiderView(tm) | 1 | 8.63 KB | 15 Jul 2008 - 05:03 |
INFOMINE VLCrawler | 1 | 13.31 KB | 21 Jul 2008 - 15:56 |
Netcraft | 1 | 0 | 24 Jul 2008 - 03:15 |
Powermarks | 1 | 0 | 01 Jul 2008 - 04:09 |
arks | 1 | 0 | 04 Jul 2008 - 17:19 |
Others | 0 | 0 | |