75 different robots | Hits | Bandwidth | Last visit |
Googlebot | 61136+232 | 37.95 GB | 31 May 2005 - 23:59 |
Unknown robot (identified by 'crawl') | 18454+637 | 9.54 GB | 31 May 2005 - 23:08 |
INFOMINE VLCrawler | 8897+29 | 7.22 GB | 30 May 2005 - 08:01 |
Yahoo Slurp | 7586+846 | 9.11 GB | 31 May 2005 - 23:53 |
MSNBot | 5749+288 | 12.08 GB | 31 May 2005 - 23:57 |
Ask | 5914+97 | 70.30 MB | 31 May 2005 - 18:15 |
ZyBorg | 3628+20 | 32.47 MB | 31 May 2005 - 23:45 |
Python-urllib | 3507+3 | 10.18 GB | 19 May 2005 - 03:49 |
BecomeBot | 2373+147 | 25.61 MB | 31 May 2005 - 11:09 |
aipbot | 773+44 | 3.79 MB | 27 May 2005 - 04:51 |
YahooSeeker Yahoo! Blog crawler | 691+38 | 3.86 MB | 31 May 2005 - 06:56 |
Grub.org | 654 | 5.38 MB | 31 May 2005 - 22:03 |
Unknown robot (identified by empty user agent string) | 529+71 | 200.92 MB | 31 May 2005 - 23:33 |
Harvest | 317 | 25.54 MB | 27 May 2005 - 12:38 |
Findlinks | 304 | 2.97 MB | 27 May 2005 - 17:06 |
Alexa (IA Archiver) | 171+52 | 260.76 MB | 31 May 2005 - 20:08 |
ConveraCrawler | 167+1 | 960.11 KB | 02 May 2005 - 06:43 |
Java (Often spam bot) | 127+6 | 2.81 MB | 31 May 2005 - 11:01 |
BaiDuSpider | 103+8 | 23.34 MB | 31 May 2005 - 16:55 |
Perl tool | 106 | 1.33 MB | 31 May 2005 - 21:04 |
Cfetch | 102 | 917.81 KB | 25 May 2005 - 10:28 |
GigaBot | 77+21 | 6.85 MB | 31 May 2005 - 03:18 |
ichiro | 94+3 | 739.45 KB | 22 May 2005 - 22:50 |
The web archive | 88+2 | 3.42 MB | 26 May 2005 - 15:33 |
IRLbot | 60+29 | 52.23 KB | 26 May 2005 - 14:54 |
Turn It In | 78+1 | 3.02 MB | 17 May 2005 - 15:55 |
Unknown robot (identified by 'robot') | 50+5 | 49.01 MB | 26 May 2005 - 17:16 |
Unknown robot (identified by 'bot*') | 42+10 | 1.32 MB | 29 May 2005 - 09:36 |
Custo | 43 | 37.34 MB | 30 May 2005 - 17:14 |
Ocelli | 34+1 | 2.01 MB | 30 May 2005 - 13:49 |
Walhello appie | 17+17 | 16.90 MB | 31 May 2005 - 03:29 |
Google AdSense | 23+8 | 21.85 MB | 24 May 2005 - 21:58 |
NutchCVS | 17+13 | 97.31 KB | 31 May 2005 - 13:25 |
Voila | 14+16 | 2.97 MB | 28 May 2005 - 10:49 |
IBM Almaden Research Center WebFountain™ | 17+11 | 2.47 MB | 29 May 2005 - 15:17 |
Yahoo-MMCrawler | 6+20 | 1.41 KB | 31 May 2005 - 06:24 |
Unknown robot (identified by hit on 'robots.txt') | 0+26 | 1.69 KB | 29 May 2005 - 19:33 |
WGet tools | 16+7 | 20.92 MB | 26 May 2005 - 19:37 |
Combine System | 12+6 | 113.98 KB | 30 May 2005 - 17:37 |
KnowItAll | 11+7 | 114.80 KB | 14 May 2005 - 00:11 |
Speedy Spider | 12+6 | 1.24 KB | 26 May 2005 - 02:42 |
CFNetwork | 16 | 118.22 MB | 31 May 2005 - 13:17 |
MSIECrawler | 12+3 | 93.44 KB | 27 May 2005 - 12:46 |
Unknown robot (identified by 'spider') | 8+4 | 21.30 KB | 30 May 2005 - 20:29 |
Scooter | 0+12 | 864 Bytes | 26 May 2005 - 13:54 |
Holmes | 0+10 | 720 Bytes | 24 May 2005 - 04:23 |
twiceler | 7+2 | 53.90 KB | 27 May 2005 - 19:48 |
Spiderman | 9 | 22.36 KB | 31 May 2005 - 15:10 |
Unknown robot (identified by 'checker') | 9 | 445.26 KB | 29 May 2005 - 09:56 |
Unknown robot (identified by 'discovery') | 8 | 0 | 27 May 2005 - 07:16 |
MJ12bot | 3+4 | 243.11 KB | 14 May 2005 - 17:57 |
NG 2.x (Exalead) | 4+2 | 51.91 KB | 26 May 2005 - 10:51 |
sohu-search | 3+2 | 561 Bytes | 31 May 2005 - 14:40 |
GoForIt.com | 2+2 | 422 Bytes | 22 May 2005 - 21:48 |
rssImagesBot | 2+2 | 422 Bytes | 11 May 2005 - 18:09 |
archive.org bot | 2+2 | 2.37 MB | 26 May 2005 - 07:39 |
psbot | 2+1 | 15.14 KB | 05 May 2005 - 00:37 |
Web Core / Roots | 3 | 26.14 KB | 24 May 2005 - 13:13 |
Cursor | 3 | 28.35 KB | 30 May 2005 - 01:55 |
HTTrack off-line browser | 3 | 961.66 KB | 17 May 2005 - 19:57 |
BBot | 3 | 139 Bytes | 23 May 2005 - 06:45 |
Fish search | 3 | 38.03 KB | 14 May 2005 - 05:00 |
Vagabondo | 1+2 | 408.12 KB | 23 May 2005 - 16:29 |
EasyDL | 3 | 31.60 KB | 13 May 2005 - 07:33 |
boitho.com-dc | 3 | 485.72 KB | 12 May 2005 - 19:39 |
UdmSearch | 1+1 | 0 | 28 May 2005 - 18:49 |
IlTrovatore-Setaccio | 1+1 | 7.24 KB | 03 May 2005 - 09:21 |
InfoSeek Robot 1.0 | 1+1 | 9.64 KB | 19 May 2005 - 11:25 |
Powermarks | 2 | 0 | 30 May 2005 - 00:46 |
Gaisbot | 1+1 | 211 Bytes | 04 May 2005 - 16:33 |
Ultraseek | 1+1 | 72 Bytes | 17 May 2005 - 12:14 |
larbin | 1+1 | 15.77 KB | 11 May 2005 - 09:26 |
Ingrid | 1 | 139 Bytes | 23 May 2005 - 16:29 |
Sunrise | 1 | 9.57 KB | 09 May 2005 - 18:37 |
Others | 0 | 7 Bytes | |