66 different robots | Hits | Bandwidth | Last visit |
bender focused_crawler | 608+91 | 2.45 GB | 31 May 2011 - 23:57 |
Yahoo Slurp | 31946+226 | 8.52 GB | 31 May 2011 - 23:55 |
Unknown robot (identified by hit on 'robots.txt') | 0+637 | 18.89 KB | 31 May 2011 - 23:50 |
Feedfetcher-Google | 3438 | 24.13 MB | 31 May 2011 - 23:49 |
Unknown robot (identified by empty user agent string) | 1209+87 | 700.86 MB | 31 May 2011 - 23:27 |
BaiDuSpider | 1290+152 | 1.60 GB | 31 May 2011 - 23:23 |
Unknown robot (identified by 'robot') | 1113+32 | 14.23 MB | 31 May 2011 - 22:42 |
Unknown robot (identified by '*bot') | 174+217 | 3.79 MB | 31 May 2011 - 22:38 |
Googlebot | 72683+95 | 42.15 GB | 31 May 2011 - 22:19 |
MSNBot-media | 1729+988 | 201.08 MB | 31 May 2011 - 21:41 |
EasyDL | 23 | 506.01 KB | 31 May 2011 - 21:38 |
Unknown robot (identified by 'bot*') | 12547+947 | 2.32 GB | 31 May 2011 - 21:00 |
FaceBook bot | 50 | 1.06 MB | 31 May 2011 - 19:39 |
WGet tools | 277 | 991.70 MB | 31 May 2011 - 19:12 |
BSpider | 0+31 | 941 Bytes | 31 May 2011 - 17:21 |
Ask | 41+15 | 2.90 MB | 31 May 2011 - 16:27 |
Yandex bot | 3407+62 | 660.68 MB | 31 May 2011 - 15:59 |
Java (Often spam bot) | 205+11 | 61.86 MB | 31 May 2011 - 14:12 |
legs | 3+4 | 70.98 KB | 31 May 2011 - 14:09 |
Exabot | 945+74 | 15.37 MB | 31 May 2011 - 13:30 |
ichiro | 327+26 | 3.48 MB | 31 May 2011 - 13:14 |
Nutch | 559+265 | 320.96 MB | 31 May 2011 - 12:20 |
Unknown robot (identified by 'crawl') | 31495+349 | 8.19 GB | 31 May 2011 - 12:03 |
Findlinks | 23+48 | 276.70 KB | 31 May 2011 - 10:00 |
Unknown robot (identified by 'spider') | 2105+72 | 154.37 MB | 31 May 2011 - 08:00 |
Speedy Spider | 36+39 | 26.60 MB | 31 May 2011 - 07:23 |
Alexa (IA Archiver) | 515+104 | 26.49 MB | 31 May 2011 - 06:05 |
Sogou Spider | 1836+15 | 708.05 MB | 31 May 2011 - 05:35 |
MJ12bot | 140+98 | 5.18 MB | 31 May 2011 - 05:21 |
WordPress | 154 | 97.93 MB | 31 May 2011 - 04:38 |
MSNBot | 399+52 | 4.45 MB | 31 May 2011 - 03:25 |
HTTrack off-line browser | 2+2 | 23.65 KB | 30 May 2011 - 02:47 |
yacy | 83+9 | 61.23 MB | 29 May 2011 - 19:12 |
Unknown robot (identified by 'discovery') | 3966+572 | 150.78 MB | 29 May 2011 - 16:51 |
CFNetwork | 49 | 520.09 MB | 29 May 2011 - 07:57 |
DoCoMo | 1341+19 | 18.38 MB | 29 May 2011 - 05:04 |
Custo | 6+6 | 6.86 MB | 28 May 2011 - 18:57 |
Perl tool | 42 | 6.27 MB | 28 May 2011 - 14:57 |
Netcraft | 8 | 56.56 KB | 28 May 2011 - 14:20 |
Sven | 2 | 1.33 MB | 28 May 2011 - 10:49 |
Jakarta commons-httpclient | 37 | 19.94 MB | 28 May 2011 - 06:47 |
Mail.ru bot | 20+1 | 4.11 MB | 28 May 2011 - 02:14 |
Python-urllib | 18 | 48.10 MB | 27 May 2011 - 16:39 |
Harvest | 22 | 2.77 MB | 27 May 2011 - 16:14 |
archive.org bot | 0+2 | 58 Bytes | 27 May 2011 - 08:20 |
Fish search | 2 | 46.81 KB | 26 May 2011 - 10:17 |
Checkbot | 5 | 0 | 26 May 2011 - 03:52 |
Motor | 30 | 66.99 MB | 24 May 2011 - 16:23 |
Voyager | 1 | 5.07 MB | 24 May 2011 - 06:55 |
WebCollage | 4 | 113.82 KB | 24 May 2011 - 01:53 |
SuperBot | 1 | 24.17 KB | 23 May 2011 - 14:30 |
MSIECrawler | 0+1 | 31 Bytes | 22 May 2011 - 09:36 |
BBot | 2 | 13.31 MB | 21 May 2011 - 23:49 |
W3C Link Checker | 2+2 | 62 Bytes | 18 May 2011 - 18:22 |
NG 1.x (Exalead) | 11 | 16.59 MB | 17 May 2011 - 22:12 |
GigaBot | 2+12 | 40.26 KB | 17 May 2011 - 13:23 |
The web archive (IA Archiver) | 0+9 | 279 Bytes | 17 May 2011 - 08:24 |
lwp-trivial | 1 | 7.07 KB | 14 May 2011 - 13:28 |
Y!J Yahoo Japan | 1 | 20.29 KB | 10 May 2011 - 01:03 |
larbin | 1+1 | 23.96 KB | 10 May 2011 - 00:17 |
VORTEX | 12 | 501.09 KB | 09 May 2011 - 00:49 |
Feedburner | 1 | 0 | 06 May 2011 - 07:35 |
UCSD Crawl | 11 | 351.82 KB | 05 May 2011 - 13:55 |
Unknown robot (identified by 'checker') | 1 | 7.07 KB | 05 May 2011 - 02:56 |
HTMLParser | 7 | 192.76 KB | 04 May 2011 - 07:55 |
LinkChecker | 0+1 | 31 Bytes | 03 May 2011 - 07:29 |