91 different robots | Hits | Bandwidth | Last visit |
Googlebot | 251080+77 | 74.63 GB | 31 May 2006 - 23:59 |
Yahoo Slurp | 33036+5115 | 26.67 GB | 31 May 2006 - 23:58 |
Unknown robot (identified by 'crawl') | 22062+654 | 15.11 GB | 31 May 2006 - 23:52 |
MSNBot | 10890+613 | 18.20 GB | 31 May 2006 - 23:44 |
Ask | 10802+99 | 157.07 MB | 31 May 2006 - 23:45 |
noxtrumbot | 3083+11 | 3.01 GB | 31 May 2006 - 10:11 |
Ocelli | 2511+2 | 1.32 GB | 27 May 2006 - 01:05 |
Nutch | 1846+35 | 17.76 MB | 31 May 2006 - 07:19 |
Heritrix | 1398+4 | 6.95 GB | 31 May 2006 - 10:34 |
GigaBot | 723+352 | 109.25 MB | 31 May 2006 - 20:15 |
BecomeBot | 871+128 | 8.59 MB | 24 May 2006 - 21:16 |
ConveraCrawler | 980+8 | 2.96 GB | 29 May 2006 - 12:34 |
Unknown robot (identified by empty user agent string) | 910+78 | 536.49 MB | 31 May 2006 - 20:28 |
Voyager | 904+22 | 10.78 MB | 31 May 2006 - 23:59 |
ksibot | 853+3 | 850.81 MB | 26 May 2006 - 08:27 |
MSIECrawler | 617+5 | 468.57 MB | 31 May 2006 - 18:21 |
Unknown robot (identified by 'robot') | 524+6 | 4.41 MB | 31 May 2006 - 19:18 |
Unknown robot (identified by 'bot*') | 428+56 | 118.84 MB | 31 May 2006 - 05:27 |
UdmSearch | 414 | 3.70 MB | 07 May 2006 - 19:45 |
Turn It In | 397+3 | 960.63 MB | 18 May 2006 - 14:17 |
Unknown robot (identified by 'spider') | 252+6 | 859.64 MB | 23 May 2006 - 07:39 |
Unknown robot (identified by hit on 'robots.txt') | 0+241 | 16.80 KB | 31 May 2006 - 14:25 |
Alexa (IA Archiver) | 210+22 | 114.40 MB | 30 May 2006 - 19:54 |
NutchCVS | 133+22 | 7.16 MB | 30 May 2006 - 02:27 |
HTTrack off-line browser | 132+3 | 4.90 MB | 31 May 2006 - 07:22 |
bender focused_crawler | 106 | 1.06 MB | 10 May 2006 - 15:15 |
Java (Often spam bot) | 60+45 | 18.09 MB | 31 May 2006 - 17:43 |
Perl tool | 94+1 | 5.07 MB | 31 May 2006 - 18:21 |
Combine System | 46+46 | 1.75 MB | 31 May 2006 - 15:44 |
BBot | 79 | 237.45 MB | 27 May 2006 - 20:41 |
MSNBot-media | 57+21 | 8.47 MB | 31 May 2006 - 17:18 |
WGet tools | 68+9 | 16.55 MB | 28 May 2006 - 17:34 |
ichiro | 46+29 | 379.44 KB | 31 May 2006 - 14:03 |
BaiDuSpider | 59+9 | 18.53 MB | 29 May 2006 - 20:40 |
lanshanbot | 41+26 | 10.05 MB | 23 May 2006 - 15:30 |
SBIder | 28+27 | 219.78 KB | 23 May 2006 - 02:57 |
Harvest | 53 | 2.09 MB | 28 May 2006 - 12:02 |
Sogou Spider | 42 | 6.98 MB | 31 May 2006 - 14:09 |
CFNetwork | 37 | 280.63 MB | 31 May 2006 - 00:53 |
aipbot | 18+18 | 728.72 KB | 31 May 2006 - 15:51 |
Unknown robot (identified by 'discovery') | 32 | 0 | 28 May 2006 - 17:14 |
Girafabot | 20+12 | 3.46 MB | 26 May 2006 - 08:01 |
psbot | 16+15 | 141.75 KB | 31 May 2006 - 14:14 |
Findlinks | 21+7 | 76.97 KB | 29 May 2006 - 20:27 |
nicebot | 28 | 267.57 KB | 31 May 2006 - 04:56 |
Spiderman | 28 | 236.21 KB | 15 May 2006 - 03:32 |
Voila | 11+14 | 4.76 MB | 22 May 2006 - 22:57 |
Yahoo-MMCrawler | 14+10 | 720 Bytes | 31 May 2006 - 22:30 |
Custo | 19 | 15.78 MB | 31 May 2006 - 07:21 |
Web Core / Roots | 17+1 | 113.75 KB | 28 May 2006 - 18:37 |
ZyBorg | 9+7 | 592.66 KB | 29 May 2006 - 05:42 |
Sina Iask Spider | 15 | 1.49 MB | 28 May 2006 - 23:15 |
Walhello appie | 8+7 | 63.28 KB | 28 May 2006 - 19:01 |
Exabot | 11+3 | 311.58 KB | 31 May 2006 - 23:57 |
Scooter | 0+12 | 864 Bytes | 28 May 2006 - 03:57 |
IBM Almaden Research Center WebFountain™ | 4+8 | 9.62 KB | 18 May 2006 - 10:09 |
larbin | 9+3 | 75.73 KB | 26 May 2006 - 20:04 |
WebCollage | 9 | 3.56 MB | 25 May 2006 - 21:35 |
Jakarta commons-httpclient | 9 | 10.34 MB | 29 May 2006 - 16:04 |
Shim-Crawler | 4+4 | 844 Bytes | 12 May 2006 - 07:33 |
BSpider | 0+8 | 576 Bytes | 18 May 2006 - 06:15 |
Gaisbot | 4+3 | 6.04 KB | 19 May 2006 - 21:34 |
ASpider (Associative Spider) | 3+3 | 633 Bytes | 24 May 2006 - 17:53 |
IRLbot | 3+3 | 46.75 KB | 31 May 2006 - 23:31 |
Unknown robot (identified by 'checker') | 6 | 314.00 KB | 26 May 2006 - 11:30 |
W3C Link Checker | 3+3 | 216 Bytes | 27 May 2006 - 19:31 |
Unknown robot (identified by 'hunter') | 5 | 236.61 KB | 16 May 2006 - 22:00 |
Google AdSense | 5 | 58.33 KB | 21 May 2006 - 13:37 |
Krugle | 2+2 | 19.28 KB | 02 May 2006 - 18:29 |
Geniebot | 2+2 | 7.86 KB | 20 May 2006 - 02:50 |
wume crawler | 3+1 | 709.83 KB | 30 May 2006 - 13:19 |
FavOrg | 3 | 16.26 KB | 05 May 2006 - 18:45 |
MJ12bot | 1+2 | 1.92 KB | 05 May 2006 - 04:27 |
LinkChecker | 3 | 16.11 KB | 23 May 2006 - 02:35 |
KnowItAll | 0+3 | 216 Bytes | 31 May 2006 - 14:17 |
Ingrid | 1+1 | 15.72 KB | 19 May 2006 - 00:56 |
DoCoMo | 2 | 13.54 KB | 13 May 2006 - 07:53 |
twiceler | 1+1 | 211 Bytes | 20 May 2006 - 21:53 |
InfoSeek Robot 1.0 | 1+1 | 9.64 KB | 28 May 2006 - 13:18 |
Powermarks | 2 | 0 | 29 May 2006 - 04:20 |
The web archive (IA Archiver) | 0+2 | 144 Bytes | 24 May 2006 - 03:35 |
UCSD Crawl | 2 | 9.32 KB | 22 May 2006 - 11:27 |
Speedy Spider | 1+1 | 211 Bytes | 05 May 2006 - 20:04 |
SuperBot | 1 | 139 Bytes | 07 May 2006 - 11:20 |
Kevin | 1 | 476.23 KB | 16 May 2006 - 06:02 |
RoboCrawl Spider | 0+1 | 72 Bytes | 30 May 2006 - 18:35 |
INFOMINE VLCrawler | 1 | 13.00 KB | 03 May 2006 - 10:39 |
StackRambler | 0+1 | 72 Bytes | 13 May 2006 - 15:56 |
Netcraft | 1 | 0 | 08 May 2006 - 04:34 |
PyQuery | 1 | 7.67 KB | 18 May 2006 - 12:44 |
Boris | 1 | 3.70 MB | 17 May 2006 - 07:39 |
Others | 0 | 4 Bytes | |