82 different robots | Hits | Bandwidth | Last visit |
Googlebot | 94503+302 | 70.07 GB | 31 Dec 2005 - 23:59 |
Unknown robot (identified by 'crawl') | 14533+561 | 15.41 GB | 31 Dec 2005 - 23:04 |
Yahoo Slurp | 10695+1318 | 7.48 GB | 31 Dec 2005 - 23:55 |
MSNBot | 11640+326 | 24.39 GB | 31 Dec 2005 - 23:42 |
Ask | 3443+144 | 226.64 MB | 31 Dec 2005 - 23:18 |
ZyBorg | 3131+3 | 27.89 MB | 31 Dec 2005 - 23:16 |
Nutch | 2528+14 | 21.85 MB | 31 Dec 2005 - 20:43 |
Jakarta commons-httpclient | 940+1354 | 56.64 MB | 30 Dec 2005 - 03:16 |
ConveraCrawler | 1043+6 | 3.35 GB | 28 Dec 2005 - 10:31 |
ichiro | 968+7 | 10.80 MB | 22 Dec 2005 - 16:46 |
Voyager | 895+25 | 9.33 MB | 31 Dec 2005 - 23:43 |
psbot | 765+51 | 6.85 MB | 29 Dec 2005 - 04:49 |
Turn It In | 680+4 | 72.17 MB | 26 Dec 2005 - 06:12 |
Unknown robot (identified by 'bot*') | 461+142 | 16.35 MB | 29 Dec 2005 - 15:19 |
Unknown robot (identified by empty user agent string) | 515+25 | 98.26 MB | 31 Dec 2005 - 21:30 |
WGet tools | 266+118 | 629.24 MB | 31 Dec 2005 - 23:57 |
MJ12bot | 332+5 | 49.27 MB | 29 Dec 2005 - 23:19 |
INFOMINE VLCrawler | 286+12 | 4.24 MB | 15 Dec 2005 - 12:07 |
Harvest | 291 | 10.48 MB | 30 Dec 2005 - 09:08 |
Java (Often spam bot) | 261+6 | 16.35 MB | 31 Dec 2005 - 14:52 |
GigaBot | 145+118 | 57.30 MB | 31 Dec 2005 - 23:31 |
BecomeBot | 179+56 | 1.41 MB | 30 Dec 2005 - 09:27 |
sohu agent | 205 | 476.97 MB | 31 Dec 2005 - 20:39 |
UdmSearch | 120 | 352.38 KB | 08 Dec 2005 - 21:49 |
NG 2.x (Exalead) | 113+5 | 732.49 KB | 22 Dec 2005 - 00:08 |
Alexa (IA Archiver) | 87+27 | 38.33 MB | 31 Dec 2005 - 18:51 |
Perl tool | 113+1 | 32.82 MB | 31 Dec 2005 - 22:31 |
SBIder | 62+27 | 398.36 KB | 29 Dec 2005 - 09:36 |
BaiDuSpider | 73+8 | 14.95 MB | 31 Dec 2005 - 23:58 |
twiceler | 77+4 | 460.97 KB | 31 Dec 2005 - 01:22 |
IBM Almaden Research Center WebFountain™ | 47+18 | 85.46 MB | 30 Dec 2005 - 03:27 |
NutchCVS | 36+22 | 1.84 MB | 30 Dec 2005 - 16:08 |
Combine System | 30+26 | 223.56 KB | 29 Dec 2005 - 07:27 |
CFNetwork | 55 | 288.71 MB | 26 Dec 2005 - 05:50 |
nicebot | 50 | 4.73 MB | 30 Dec 2005 - 09:39 |
Findlinks | 24+23 | 271.94 KB | 31 Dec 2005 - 18:03 |
G2Crawler | 28+13 | 420.74 KB | 29 Dec 2005 - 05:47 |
sohu-search | 22+11 | 1.85 MB | 13 Dec 2005 - 15:49 |
Voila | 16+17 | 337.67 KB | 31 Dec 2005 - 16:31 |
noxtrumbot | 31+1 | 279.30 KB | 19 Dec 2005 - 17:10 |
Unknown robot (identified by 'robot') | 29+3 | 108.57 KB | 30 Dec 2005 - 19:14 |
Unknown robot (identified by hit on 'robots.txt') | 0+31 | 2.04 KB | 30 Dec 2005 - 16:17 |
WebCollage | 29 | 8.80 MB | 30 Dec 2005 - 07:37 |
Unknown robot (identified by 'discovery') | 25 | 0 | 25 Dec 2005 - 05:02 |
MSIECrawler | 20+4 | 1.59 MB | 23 Dec 2005 - 16:36 |
Girafabot | 16+7 | 2.45 MB | 28 Dec 2005 - 23:35 |
Unknown robot (identified by 'spider') | 10+10 | 2.06 KB | 30 Dec 2005 - 03:08 |
Argus | 12+6 | 869.09 KB | 26 Dec 2005 - 01:01 |
Geniebot | 8+8 | 34.24 KB | 29 Dec 2005 - 14:47 |
MSRBOT | 15+1 | 148.15 KB | 14 Dec 2005 - 20:48 |
Google AdSense | 13+2 | 704.44 KB | 23 Dec 2005 - 05:12 |
Yahoo-MMCrawler | 3+12 | 16.61 KB | 31 Dec 2005 - 19:40 |
HTTrack off-line browser | 13 | 1.41 MB | 28 Dec 2005 - 23:16 |
Gaisbot | 7+5 | 647.67 KB | 18 Dec 2005 - 20:13 |
Walhello appie | 6+6 | 23.98 KB | 24 Dec 2005 - 16:56 |
Custo | 10 | 6.36 MB | 30 Dec 2005 - 22:42 |
larbin | 5+4 | 49.59 KB | 31 Dec 2005 - 16:55 |
Unknown robot (identified by 'checker') | 8 | 386.20 KB | 31 Dec 2005 - 15:25 |
lwp-trivial | 8 | 20.12 KB | 25 Dec 2005 - 19:31 |
archive.org bot | 4+4 | 4.75 MB | 30 Dec 2005 - 22:20 |
boitho.com-dc | 3+4 | 1.17 MB | 25 Dec 2005 - 01:17 |
Web Core / Roots | 5+1 | 36.35 KB | 25 Dec 2005 - 11:51 |
Pompos | 2+4 | 566 Bytes | 30 Dec 2005 - 06:39 |
Spiderman | 5 | 27.09 KB | 25 Dec 2005 - 13:23 |
xirq | 2+2 | 422 Bytes | 17 Dec 2005 - 05:16 |
Vagabondo | 2+2 | 422 Bytes | 14 Dec 2005 - 07:14 |
Cursor | 3 | 31.00 KB | 03 Dec 2005 - 07:50 |
Python-urllib | 3 | 29.05 KB | 02 Dec 2005 - 00:17 |
Unknown robot (identified by '*bot') | 3 | 23.86 KB | 28 Dec 2005 - 11:19 |
HTMLParser | 3 | 9.57 KB | 26 Dec 2005 - 11:35 |
VORTEX | 0+3 | 216 Bytes | 26 Dec 2005 - 23:52 |
Sina Iask Spider | 2 | 13.35 KB | 31 Dec 2005 - 11:24 |
Ingrid | 1+1 | 15.72 KB | 18 Dec 2005 - 18:47 |
Netcraft | 2 | 0 | 19 Dec 2005 - 21:14 |
Powermarks | 2 | 0 | 09 Dec 2005 - 04:05 |
LinkChecker | 2 | 611.11 KB | 16 Dec 2005 - 12:46 |
SynooBot | 1+1 | 211 Bytes | 12 Dec 2005 - 13:41 |
KnowItAll | 1+1 | 37.07 KB | 15 Dec 2005 - 18:35 |
Crawl libcrawl | 0+1 | 72 Bytes | 20 Dec 2005 - 12:26 |
MapoftheInternet.com | 0+1 | 72 Bytes | 13 Dec 2005 - 23:48 |
Others | 0 | 2.49 KB | |