| 93 different robots | Hits | Bandwidth | Last visit |
|---|
| Googlebot | 90428+318 | 43.45 GB | 31 Oct 2005 - 23:59 |
| Unknown robot (identified by 'crawl') | 70243+591 | 57.55 GB | 31 Oct 2005 - 22:09 |
| Yahoo Slurp | 11401+1380 | 7.86 GB | 31 Oct 2005 - 23:56 |
| Ask | 7562+159 | 113.53 MB | 31 Oct 2005 - 23:33 |
| MSNBot | 6046+251 | 10.10 GB | 31 Oct 2005 - 21:33 |
| BecomeBot | 3856+159 | 37.74 MB | 31 Oct 2005 - 19:47 |
| Accoona-AI-Agent | 2682+4 | 3.38 GB | 29 Oct 2005 - 02:07 |
| Nutch | 2524+7 | 21.70 MB | 21 Oct 2005 - 21:14 |
| ZyBorg | 2107+5 | 19.63 MB | 31 Oct 2005 - 13:01 |
| Jakarta commons-httpclient | 1321+3 | 27.80 MB | 28 Oct 2005 - 10:31 |
| The web archive | 747+27 | 504.00 MB | 25 Oct 2005 - 17:51 |
| aipbot | 651+13 | 4.45 MB | 26 Oct 2005 - 16:29 |
| GigaBot | 511+77 | 5.08 MB | 27 Oct 2005 - 13:00 |
| sohu agent | 573 | 7.02 MB | 31 Oct 2005 - 15:07 |
| ConveraCrawler | 569+2 | 1.07 GB | 29 Oct 2005 - 07:25 |
| NutchCVS | 299+131 | 28.79 MB | 28 Oct 2005 - 21:12 |
| Unknown robot (identified by empty user agent string) | 377+35 | 175.39 MB | 31 Oct 2005 - 16:04 |
| ichiro | 403+8 | 4.30 MB | 31 Oct 2005 - 22:28 |
| Turn It In | 368+2 | 834.18 MB | 30 Oct 2005 - 12:05 |
| VSE | 353+3 | 4.61 MB | 31 Oct 2005 - 05:29 |
| Harvest | 342 | 44.95 MB | 30 Oct 2005 - 12:04 |
| psbot | 284+38 | 2.31 MB | 31 Oct 2005 - 22:15 |
| Cfetch | 257+36 | 2.60 MB | 31 Oct 2005 - 23:39 |
| nicebot | 250 | 3.28 MB | 31 Oct 2005 - 17:14 |
| Perl tool | 213 | 27.76 MB | 31 Oct 2005 - 19:27 |
| RufusBot Rufus Web Miner | 109+8 | 1021.00 KB | 31 Oct 2005 - 18:13 |
| Grub.org | 101 | 854.73 KB | 11 Oct 2005 - 23:18 |
| Unknown robot (identified by 'bot*') | 62+25 | 23.34 MB | 31 Oct 2005 - 22:50 |
| yacy | 67+1 | 826.12 KB | 17 Oct 2005 - 13:14 |
| WGet tools | 58+3 | 107.05 MB | 31 Oct 2005 - 19:54 |
| IBM Almaden Research Center WebFountain™ | 32+22 | 2.48 MB | 30 Oct 2005 - 22:36 |
| Custo | 44 | 27.30 MB | 31 Oct 2005 - 14:18 |
| MJ12bot | 41+3 | 335.21 KB | 17 Oct 2005 - 14:22 |
| Unknown robot (identified by 'discovery') | 40 | 0 | 31 Oct 2005 - 11:41 |
| Unknown robot (identified by hit on 'robots.txt') | 0+34 | 2.39 KB | 31 Oct 2005 - 17:24 |
| BaiDuSpider | 28+5 | 6.59 MB | 28 Oct 2005 - 17:40 |
| Voila | 15+18 | 683.60 KB | 24 Oct 2005 - 00:43 |
| CFNetwork | 30 | 125.91 MB | 31 Oct 2005 - 10:26 |
| sohu-search | 20+10 | 1.80 MB | 31 Oct 2005 - 15:39 |
| Unknown robot (identified by 'spider') | 14+14 | 2.75 KB | 31 Oct 2005 - 03:44 |
| TITAN | 25 | 32.07 MB | 08 Oct 2005 - 04:44 |
| MSIECrawler | 19+6 | 12.38 MB | 31 Oct 2005 - 02:16 |
| NG 2.x (Exalead) | 21+2 | 235.27 KB | 24 Oct 2005 - 01:14 |
| Girafabot | 16+7 | 2.27 MB | 30 Oct 2005 - 02:44 |
| Java (Often spam bot) | 21+2 | 799.26 KB | 31 Oct 2005 - 05:38 |
| Unknown robot (identified by 'robot') | 21+2 | 20.87 KB | 31 Oct 2005 - 15:56 |
| Google AdSense | 19+3 | 165.97 KB | 29 Oct 2005 - 18:20 |
| Alexa (IA Archiver) | 12+9 | 203.71 KB | 30 Oct 2005 - 01:13 |
| Findlinks | 12+8 | 237.82 KB | 31 Oct 2005 - 20:06 |
| Combine System | 12+8 | 117.89 KB | 31 Oct 2005 - 04:44 |
| SBIder | 8+9 | 43.85 KB | 06 Oct 2005 - 21:58 |
| Walhello appie | 7+8 | 284.82 KB | 30 Oct 2005 - 20:16 |
| Argus | 10+5 | 776.91 KB | 27 Oct 2005 - 00:06 |
| Gaisbot | 6+6 | 19.13 KB | 30 Oct 2005 - 16:03 |
| Python-urllib | 8+1 | 7.66 MB | 30 Oct 2005 - 06:32 |
| Metaspinner | 4+4 | 844 Bytes | 30 Oct 2005 - 10:13 |
| Yahoo-MMCrawler | 4+4 | 16.35 KB | 30 Oct 2005 - 04:27 |
| archive.org bot | 4+4 | 4.75 MB | 27 Oct 2005 - 07:12 |
| Spiderman | 7 | 22.08 KB | 26 Oct 2005 - 05:19 |
| IlTrovatore-Setaccio | 3+3 | 633 Bytes | 28 Oct 2005 - 17:43 |
| Cursor | 6 | 29.40 KB | 29 Oct 2005 - 17:37 |
| twiceler | 3+3 | 4.72 KB | 26 Oct 2005 - 03:25 |
| Unknown robot (identified by 'checker') | 6 | 386.35 KB | 28 Oct 2005 - 14:07 |
| UCSD Crawl | 6 | 629.81 KB | 01 Oct 2005 - 17:10 |
| Scooter | 0+5 | 360 Bytes | 15 Oct 2005 - 13:42 |
| DoCoMo | 2+2 | 422 Bytes | 14 Oct 2005 - 23:41 |
| HTTrack off-line browser | 3+1 | 211 Bytes | 20 Oct 2005 - 04:25 |
| IRLbot | 3+1 | 489 Bytes | 21 Oct 2005 - 20:41 |
| Netcraft | 4 | 0 | 24 Oct 2005 - 10:06 |
| KnowItAll | 2+2 | 15.21 KB | 21 Oct 2005 - 20:38 |
| lwp-trivial | 4 | 24.71 KB | 30 Oct 2005 - 11:10 |
| larbin | 2+2 | 12.41 KB | 27 Oct 2005 - 23:25 |
| boitho.com-dc | 2+1 | 24.03 KB | 08 Oct 2005 - 14:40 |
| Unknown robot (identified by 'hunter') | 3 | 35.86 KB | 18 Oct 2005 - 13:15 |
| Aport | 3 | 31.81 KB | 28 Oct 2005 - 07:18 |
| Computer and Automation Research Institute Crawler | 1+1 | 211 Bytes | 26 Oct 2005 - 03:16 |
| Ingrid | 1+1 | 211 Bytes | 14 Oct 2005 - 02:30 |
| INFOMINE VLCrawler | 1+1 | 211 Bytes | 04 Oct 2005 - 16:28 |
| Asterias | 1+1 | 211 Bytes | 20 Oct 2005 - 17:16 |
| Powermarks | 2 | 0 | 24 Oct 2005 - 00:43 |
| The web archive (IA Archiver) | 0+2 | 144 Bytes | 06 Oct 2005 - 07:22 |
| Vagabondo | 1+1 | 8.29 KB | 22 Oct 2005 - 20:49 |
| YahooSeeker Yahoo! Blog crawler | 2 | 562.80 KB | 15 Oct 2005 - 12:54 |
| NutchOSU-VLIB | 1+1 | 211 Bytes | 08 Oct 2005 - 10:10 |
| MojeekBot | 1+1 | 211 Bytes | 08 Oct 2005 - 04:54 |
| Hylanda | 1+1 | 211 Bytes | 08 Oct 2005 - 01:07 |
| wume crawler | 1+1 | 211 Bytes | 26 Oct 2005 - 17:32 |
| Webdup | 1 | 4.24 KB | 29 Oct 2005 - 19:17 |
| updated | 1 | 139 Bytes | 04 Oct 2005 - 17:35 |
| Orbiter | 1 | 139 Bytes | 12 Oct 2005 - 06:58 |
| Miva | 0+1 | 72 Bytes | 10 Oct 2005 - 21:02 |
| Others | 0 | 4.36 KB | |