USER-AGENT: * Disallow: /images/ Disallow: /test/ Disallow: /cgi-bin/ USER-AGENT: CherryPickerElite/1.0 Disallow: / USER-AGENT: CherryPickerSE/1.0 Disallow: / USER-AGENT: CherryPicker Disallow: / USER-AGENT: EmailCollector Disallow: / USER-AGENT: EmailSiphon Disallow: / USER-AGENT: EmailWolf Disallow: / USER-AGENT: ExtractorPro Disallow: / USER-AGENT: Harvest/1.5 Disallow: / USER-AGENT: ia_archiver Disallow: / USER-AGENT: ia_archiver/1.6 Disallow: / USER-AGENT: LinkextractorPro Disallow: / USER-AGENT: Offline Explorer Disallow: / USER-AGENT: SiteSnagger Disallow: / USER-AGENT: Speedy Spider Disallow: / USER-AGENT: TeleportPro Disallow: / USER-AGENT: Teleport Disallow: / USER-AGENT: WebZip/4.0 Disallow: / USER-AGENT: WebZip Disallow: / USER-AGENT: WebStripper Disallow: / USER-AGENT: WebCopier Disallow: / USER-AGENT: WebBandit/3.50 Disallow: / USER-AGENT: Wget/1.6 Disallow: / USER-AGENT: Wget/1.5.3 Disallow: / USER-AGENT: Wget Disallow: / USER-AGENT: Web Image Collector Disallow: / USER-AGENT: webreaper Disallow: /