Bots von Backlink Checker Tools с блокировкой .htaccess

1. ПереписатьCond или SetEnvIfNoCase
2. Агенты пользователей von bekannten Bots
3. Ресурсы

Es gibt gute Gründe zu verhindern, dass Links vin eigene Seiten in Backlink Profil Анализ инструментов SEO (Linkresarchtools) с помощью инструментов, OpenSiteExplorer, Majestic SEO, Sistrix, Searchmetrics, SeoKicks, SEMRush или Xovi angezeigt wer.

Man kann die Scraper Bots (часто Spider oder Crawler genannt) zwar mittels robots.txt aussperren, das hat jedoch den Nachteil, dass die for für jederman öffentlich ersichtlich ист.

ПереписатьCond или SetEnvIfNoCase

Дешевый набор данных о Хольцхакерском методе mittels mod_rewrite in der .htaccess Datei ein. Dabei gibt es zwei Möglichkeiten: Entweder с «Переписать условия» или «SetEnvIfNoCase».

Während bei der Lösung mit SetEnvIfNoCase am Schluss mit «deny from» der Bot mit einem 403 (запрещено) geblockt wird, kann man beim mod_rewrite Ансатц ден Бот бейшпильсвайз цу эйнер Зайт-дю-Конкренренбрюкен-де-Линкен, Германия назад einfach das Nichtvorhandensein der Seite mittels 404 vortäuschen.

Letzteres erreicht man mit folgendem Код в дер .htaccess Дата:

RewriteEngine On RewriteCond% {HTTP_USER_AGENT} ^. * AhrefsBot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Alexibot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Aqua_Products. * $ NC , OR] RewriteCond% {HTTP_USER_AGENT} ^. * Asterias. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * B2w / 0.1. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * BackDoorBot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * BacklinkCrawler. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * BlackWidow. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * BlekkoBot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * BlowFish. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Закладка \ search \ tool. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * BotALot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Bot \ mailto: [email protected]. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * BuiltBotTough. * $ [ NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Яблочко. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * BunnySlippers. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * CheeseBot. * $ [NC, OR] R ewriteCond% {HTTP_USER_AGENT} ^. * CherryPicker. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * ChinaClaw. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Коперник. * $ [NC, ИЛИ ] RewriteCond% {HTTP_USER_AGENT} ^. * CopyRightCheck. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Cosmos. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Crescent. * $ [NC, ИЛИ] RewriteCond% {HTTP_USER_AGENT} ^. * Custo. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * DISCo. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * DittoSpyder. * $ [NC , OR] RewriteCond% {HTTP_USER_AGENT} ^. * Dotbot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Скачать \ Demon. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Dumbot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * ECatch. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * EirGrabber. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Электронная почта. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * EmailCollector. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * EmailSiphon. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * EmailWolf . * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Enterprise_Search. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * EroCrawler. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Exabot. * $ [NC, OR] RewriteCond % {HTTP_USER_AGENT} ^. * Express \ WebPictures. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * ExtractorPro. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * EyeNetIE. * $ [NC, ИЛИ] RewriteCond% {HTTP_USER_AGENT} ^. * FairAd \ Client. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Flaming \ AttackBot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * FlashGet. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Foobot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Gaisbot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * GetRight . * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * GetWeb!. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Gigabot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Go! Zilla. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Go-Ahead-Got-It. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * GrabNet. * $ [NC, ИЛИ] RewriteCond% {HTTP_USER_AGENT} ^. * Графула. * $ [NC, ИЛИ] RewriteCond% {HTTP_USER_AGENT} ^. * Grub. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Harvest. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Hatena \ Antenna. * $ [NC , OR] RewriteCond% {HTTP_USER_AGENT} ^. * Hloader. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * HMView. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Httplib. * $ [ NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * HTTrack. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Humanlinks. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Ia_archiver. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Image \ Stripper. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Image \ Sucker. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Indy \ Library. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * InfoNaviRobot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * InterGET. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT } ^. * Internet \ Ninja. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Iron33. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * JennyBot. * $ [NC, OR] RewriteCond % {HTTP_USER_AGENT} ^. * Jetbot. * $ [NC, OR] R ewriteCond% {HTTP_USER_AGENT} ^. * JetCar. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * JOC \ Web \ Spider. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Kenjin. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Keyword. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Larbin. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * LeechFTP. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * LexiBot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * LibWeb / clsHTTP. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * LinkextractorPro. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * LinkScan. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * LinkWalker. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * LNSpiderguy. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Lookmart. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Lwp-тривиально. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT } ^. * Marketing. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Mass \ Downloader. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Мата \ Хари. * $ [NC, OR ] RewriteCond% {HTTP_USER_AGENT} ^. * Microsoft \ URL \ Control. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * MIDown \ tool. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * MIIxpc. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Mister \ PiX. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * MJ12bot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Moget. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * MSIECrawler. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Naver. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Navroad. * $ [NC, OR] RewriteCond % {HTTP_USER_AGENT} ^. * NearSite. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * NetAnts. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * NetMechanic. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * NetSpider. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * NetZIP. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Net \ Vampire. * $ [NC , OR] RewriteCond% {HTTP_USER_AGENT} ^. * NICErsPRO. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Nutch. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Осьминог. * $ [ NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Не в сети \ Explorer. * $ [NC, O R] RewriteCond% {HTTP_USER_AGENT} ^. * Offline \ Navigator. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Openbot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Openfind. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Oracle \ Ultra \ Search. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Owlinbot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * PageGrabber. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Папа \ Foto. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Pavuk. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT } ^. * pcBrowser. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * PerMan. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * ProPowerBot. * $ [NC, OR] RewriteCond% { HTTP_USER_AGENT} ^. * ProWebWalker. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Proximic. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Psbot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Python-urllib. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * QueryN. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Радиация \ Retriever \ 1.1. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^ . * RealDownload. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * ReGet. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * RMA. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * rogerbot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * scooter. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Screaming. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT } ^. * SearchEngineWorld. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * SearchmetricsBot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Searchpreview. * $ [NC, OR] RewriteCond% { HTTP_USER_AGENT} ^. * Semrush. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * SEO. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Seolytics. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Sistrix. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * SiteSnagger. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * SmartDownload. * $ [NC, OR] RewriteCond % {HTTP_USER_AGENT} ^. * Sootle. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * SpankBot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Spanner. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Spy. * $ [N C, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Stanford. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * SuperBot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * SuperHTTP. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Surfbot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * SurveyBot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Suzuran. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Szukacz. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * TAkeOut. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Телепорт. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Telesoft. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * TheNomad. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * The \ Intraformant. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * ToCrawl / UrlDispatcher. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * True_Robot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT } ^. * turingos. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Updownerbot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * URLy \ Warning. * $ [NC, OR] RewriteCond % {HTTP_USER_AGENT} ^. * URL_Spider_P ro. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * VCI. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * VoidEYE. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebAuto. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebBandit. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebCopier. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^ . * WebEnhancer. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebFetch. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebGo \ IS. * $ [NC, OR] RewriteCond% { HTTP_USER_AGENT} ^. * WebLeacher. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebmasterWorld. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebReaper. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebSauger. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Веб-сайт. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Веб-сайт \ eXtractor. * $ [NC, OR ] RewriteCond% {HTTP_USER_AGENT} ^. * Веб-сайт \ Quester. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Вебстер. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebStripper. * $ [ NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebVac . * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebViewer. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebWhacker. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebZip. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WebZIP. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Web \ Image \ Collector. * $ [NC, OR] RewriteCond% { HTTP_USER_AGENT} ^. * Web \ Sucker. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Wget. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Вдова. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WWW-Collector-E. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * WWWOFFLE. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Xaldon \ WebSpider. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Xenu. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * XoviBot. * $ [NC, OR] RewriteCond% {HTTP_USER_AGENT} ^. * Зевс . * $ [NC] RewriteRule ^ - [L, R = 404]

Агенты пользователей von bekannten Bots

Doch was sind denn nun die User Agent strings der Crawler-Bots der populärsten Проверка обратных ссылок? Glücklicherweise geben die meisten Anbieter von Backlink Инструменты diese Информация mehr (Ahrefs, MOZ) или weniger (Majestic SEO, Sistrix) bereitwillig bekannt:

www.ahrefs.com: AhrefsBot ( Quelle )
www.backlinktest.com: BacklinkCrawler ( Quelle )
www.linkresearchtools.com (Keine Hinweise Gefunden)
www.majesticseo.com: MJ12bot ( Quelle )
www.opensiteexplorer.org: rogerbot ( Quelle )
www.screamingfrog.co.uk: Screaming Frog SEO Spider ( Quelle )
www.searchmetrics.com: SearchmetricsBot ( Quelle )
www.semrush.com: SemrushBot ( Quelle )
www.seokicks.de: SEOkicks-Robot ( Quelle )
www.seolytics.de: seolytics (keine Hinweise gefunden)
www.seo-united.de: (Кейн Хайнвейз Гефунден)
www.sistrix.de: sistrix ( Quelle )
www.xovi.de: XoviBot ( Quelle )

Ресурсы

Рукавицы пользователей-агентов в блоке «Переписать условия» или «SetEnvIfNoCase»: http://en.linuxreviews.org/HOWTO_stop_automated_spam-bots_using_.htaccess
Пользователь-агент Stets aktualisierte Liste von allen bekannten Боты: http://user-agent-string.info/list-of-ua/bots

Похожие

Комментарии