Содержит ошибки
Файл управления сканированием сайта роботами
?
Robots.txt указывает поисковым роботам, какие страницы сканировать, а какие — нет. Ошибки в файле могут случайно закрыть важные разделы от индексации.
Проверяемая страница запрещена к индексации в robots.txt для Яндекса. Исправьте файл robots.txt!
Цепочка редиректов для файла robots.txt:
http://avanti24.pl/robots.txt
301 MovedPermanently
https://avanti24.pl/robots.txt
200 OK
Показать содержимое robots.txt
#Zastrze�enie prawnoautorskie
#Wersja polska:
#W�a�ciciel niniejszego serwisu internetowego korzystaj�c z uprawnienia, przyznanego przez art. 4 ust. 3 Dyrektywy Parlamentu Europejskiego i Rady (UE) 2019/790 z dnia 17 kwietnia 2019 r. w sprawie prawa autorskiego i praw pokrewnych na jednolitym rynku cyfrowym oraz zmiany dyrektyw 96/9/WE i 2001/29/WE (Dz. U. UE. L. z 2019 r. Nr 130, str. 92) nie wyra�a zgody na zwielokrotnianie, wykorzystywanie lub przechowywanie jakichkolwiek tre�ci w postaci tekst�w i danych oraz program�w komputerowych i baz danych dost�pnych w serwisie internetowym, w celu ich eksploracji polegaj�cej na analizie, r�wnie� przy zastosowaniu zautomatyzowanych technik, d���cej do wygenerowania informacji obejmuj�cych w szczeg�lno�ci wzorce, tendencje i korelacje. Powy�sze zastrze�enie nie dotyczy eksploracji tre�ci dokonywanej w celu umieszczania stron internetowych w ramach wynik�w wyszukiwania, jak te� nie ma na celu i nie mo�e by� interpretowane jako wy��czenie lub ograniczenie uprawnie� konsument�w do korzystania z tre�ci w ramach udzielonego zezwolenia lub dozwolonego u�ytku.
#Uzyskaj wi�cej informacji w Zastrze�eniu dot. eksploracji tekst�w i danych pod adresem agora.pl/zastrzezenie-prawne
#Wersja angielska:
#The owner of this website, in exercising of the right granted by Article 4(3) of Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the digital single market and amending Directives 96/9/EC and 2001/29/EC (OJ EU. L. 2019 No. 130, p. 92) does not authorize reproduction, use or storage of any content in the form of texts and data, computer programs and databases available on the website for the purpose of exploring them by means of analysis, also using automated techniques, with the aim of generating information, including in particular patterns, trends and correlations. The above disclaimer does not apply to content mining carried out for the purpose of placing websites within search results, nor is it intended to or may be construed as an exclusion or limitation of the rights of consumers to use content within the scope of the granted authorization or permissible use.
#More information can be found in Reservation on text and data mining at agora.pl/en/copyright-reservation
#Koniec zastrze�enia prawnoautorskiego
User-agent: *
Disallow: /ot-amp-consent
Disallow: /*/wyszukaj/
Disallow: /*servlet
Disallow: /reloadwww?
Disallow: /dfptools/adview/
Disallow: /pub/ips/*
Disallow: /ods?
Disallow: /getFile.servlet*
Disallow: /aliasy/blad.jsp
Disallow: /znajdz.do
Disallow: /portalSearch.do
Disallow: /im/ab/b4/10/z17515435Q.jpg
Disallow: /75224259/
User-agent: Googlebot-News
Disallow: /nowy/
Disallow: /mapa_strony
Disallow: /*/wyszukaj/
Disallow: /*/51,
Disallow: /*/55,
Disallow: /*/2,
Disallow: /*order=
Disallow: /*obxx=
Disallow: /*tag=
Disallow: /reloadwww?
Disallow: /ods?
Disallow: /*servlet
Disallow: /dfptools/adview/
# START 2023-10-30
User-agent: 360Spider
Disallow: /
User-agent: 80legs
Disallow: /
User-agent: Aboundex
Disallow: /
User-agent: accelobot
Disallow: /
User-agent: Add\ Catalog
Disallow: /
User-agent: aiHitBot
Disallow: /
User-agent: Alexibot
Disallow: /
User-agent: Aqua_Products
Disallow: /
User-agent: AskJeeves
Disallow: /
User-agent: asterias
Disallow: /
User-agent: awcheckBot
Disallow: /
User-agent: b2w/0.1
Disallow: /
User-agent: BackDoorBot/1.0
Disallow: /
User-agent: BacklinkCrawler
Disallow: /
User-agent: Baiduspider
Disallow: /
User-agent: Barkrowler
Disallow: /
User-agent: BecomeBot
Disallow: /
User-agent: BLEXBot
Disallow: /
User-agent: BlowFish/1.0
Disallow: /
User-agent: Bookmark search tool
Disallow: /
User-agent: BotALot
Disallow: /
User-agent: brandwatch.net
Disallow: /
User-agent: BuiltBotTough
Disallow: /
User-agent: Bullseye/1.0
Disallow: /
User-agent: BunnySlippers
Disallow: /
User-agent: Butterfly
Disallow: /
User-agent: CatchBot
Disallow: /
User-agent: Charlotte
Disallow: /
User-agent: CheeseBot
Disallow: /
User-agent: CherryPicker
Disallow: /
User-agent: CherryPickerElite/1.0
Disallow: /
User-agent: CherryPickerSE/1.0
Disallow: /
User-agent: Clickagy Intelligence Bot v2
Disallow: /
User-agent: CLIPish
Disallow: /
User-agent: Cliqzbot
Disallow: /
User-agent: COMODO
Disallow: /
User-agent: Comodo-Certificates-Spider
Disallow: /
User-agent: CompSpyBot
Disallow: /
User-agent: Copernic
Disallow: /
User-agent: CopyRightCheck
Disallow: /
User-agent: cosmos
Disallow: /
User-agent: crawler
Disallow: /
User-agent: Crescent
Disallow: /
User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
Disallow: /
User-agent: Curious
Disallow: /
User-agent: curl
Disallow: /
User-agent: DataForSeoBot
Disallow: /
User-agent: dataprovider\.com
Disallow: /
User-agent: DinoPing
Disallow: /
User-agent: discoverybot
Disallow: /
User-agent: DittoSpyder
Disallow: /
User-agent: DomainCrawler
Disallow: /
User-agent: dotbot
Disallow: /
User-agent: dotnetdotcom
Disallow: /
User-agent: Dow\ Jones\ Searchbot
Disallow: /
User-agent: dumbot
Disallow: /
User-agent: EasouSpider
Disallow: /
User-agent: EmailCollector
Disallow: /
User-agent: EmailSiphon
Disallow: /
User-agent: EmailWolf
Disallow: /
User-agent: Enterprise_Search
Disallow: /
User-agent: Enterprise_Search/1.0
Disallow: /
User-agent: EroCrawler
Disallow: /
User-agent: es
Disallow: /
User-agent: Exabot
Disallow: /
User-agent: ExtractorPro
Disallow: /
User-agent: EyeMonIT
Disallow: /
User-agent: EzineArticlesLinkScanner
Disallow: /
User-agent: Ezooms
Disallow: /
User-agent: FairAd Client
Disallow: /
User-agent: Feedpresso Content Index Bot
Disallow: /
User-agent: Flaming AttackBot
Disallow: /
User-agent: Foobot
Disallow: /
User-agent: FreeFind
Disallow: /
User-agent: FTRF\:\ Friendly
Disallow: /
User-agent: Gaisbot
Disallow: /
User-agent: GetRight/4.2
Disallow: /
User-agent: gigabot
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: grub
Disallow: /
User-agent: grub-client
Disallow: /
User-agent: Harvest/1.5
Disallow: /
User-agent: Hatena Antenna
Disallow: /
User-agent: hloader
Disallow: /
User-agent: http://linkfluence.com
Disallow: /
User-agent: http://mediacloud.org
Disallow: /
User-agent: http://www.SearchEngineWorld.com bot
Disallow: /
User-agent: http://www.WebmasterWorld.com bot
Disallow: /
User-agent: HTTP_Request
Disallow: /
User-agent: HTTP_Request2
Disallow: /
User-agent: httplib
Disallow: /
User-agent: https://developers.snap.com/robots
Disallow: /
User-agent: https://help.qwant.com/bot/
Disallow: /
User-agent: humanlinks
Disallow: /
User-agent: ia_archiver
Disallow: /
User-agent: ia_archiver/1.6
Disallow: /
User-agent: Indy\ Library
Disallow: /
User-agent: InfoNaviRobot
Disallow: /
User-agent: ip\-web\-crawler\.com
Disallow: /
User-agent: Iron33/1.0.2
Disallow: /
User-agent: Jakarta\ Commons-HttpClient
Disallow: /
User-agent: Jeeves
Disallow: /
User-agent: JennyBot
Disallow: /
User-agent: Jetbot
Disallow: /
User-agent: Jetbot/1.0
Disallow: /
User-agent: JikeSpider
Disallow: /
User-agent: Kenjin Spider
Disallow: /
User-agent: Keyword Density/0.9
Disallow: /
User-agent: larbin
Disallow: /
User-agent: LexiBot
Disallow: /
User-agent: libWeb/clsHTTP
Disallow: /
User-agent: libwww-perl
Disallow: /
User-agent: lindex\.com
Disallow: /
User-agent: Linguee
Disallow: /
User-agent: linkdex\.com
Disallow: /
User-agent: linkdexbot
Disallow: /
User-agent: LinkextractorPro
Disallow: /
User-agent: LinkScan/8.1a Unix
Disallow: /
User-agent: LinkWalker
Disallow: /
User-agent: lipperhey
Disallow: /
User-agent: LNSpiderguy
Disallow: /
User-agent: looksmart
Disallow: /
User-agent: ltbot
Disallow: /
User-agent: lwp-trivial
Disallow: /
User-agent: lwp-trivial/1.34
Disallow: /
User-agent: Lynx
Disallow: /
User-agent: magpie\-crawler
Disallow: /
User-agent: Mata Hari
Disallow: /
User-agent: Mattermost-Bot
Disallow: /
User-agent: http://MegaIndex.ru
Disallow: /
User-agent: Microsoft URL Control
Disallow: /
User-agent: Microsoft URL Control - 5.01.4511
Disallow: /
User-agent: Microsoft URL Control - 6.00.8169
Disallow: /
User-agent: MIIxpc
Disallow: /
User-agent: MIIxpc/4.2
Disallow: /
User-agent: Mister PiX
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: moget
Disallow: /
User-agent: moget/2.1
Disallow: /
User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
Disallow: /
User-agent: Mozilla/5.0 (compatible; Googlebot/2.1; startmebot/1.0; +https://start.me/bot)
Disallow: /
User-agent: Mozilla/5.0 (compatible; ImagesiftBot; +imagesift.com)
Disallow: /
User-agent: MSIE\ or\ Firefox\ mutant
Disallow: /
User-agent: MSIECrawler
Disallow: /
User-agent: naver
Disallow: /
User-agent: NCBot
Disallow: /
User-agent: NetAnts
Disallow: /
User-agent: NetcraftSurveyAgent
Disallow: /
User-agent: netEstate\ NE\ Crawler
Disallow: /
User-agent: Neticle Crawler
Disallow: /
User-agent: NetMechanic
Disallow: /
User-agent: Netseer
Disallow: /
User-agent: NextGenSearchBot
Disallow: /
User-agent: NICErsPRO
Disallow: /
User-agent: Nutch
Disallow: /
User-agent: Ocelli
Disallow: /
User-agent: Offline Explorer
Disallow: /
User-agent: OmniExplorer_Bot
Disallow: /
User-agent: Openbot
Disallow: /
User-agent: Openfind
Disallow: /
User-agent: Openfind data gathere
Disallow: /
User-agent: OpenWebIndex
Disallow: /
User-agent: Oracle Ultra Search
Disallow: /
User-agent: PagesInventory
Disallow: /
User-agent: PEAR
Disallow: /
User-agent: PeoplePal
Disallow: /
User-agent: PerMan
Disallow: /
User-agent: PetalBot
Disallow: /
User-agent: ProCogSEOBot
Disallow: /
User-agent: ProPowerBot/2.14
Disallow: /
User-agent: ProWebWalker
Disallow: /
User-agent: proximic
Disallow: /
User-agent: psbot
Disallow: /
User-agent: PubMatic Crawler Bot
Disallow: /
User-agent: purebot
Disallow: /
User-agent: QueryN Metasearch
Disallow: /
User-agent: QuerySeekerSpider
Disallow: /
User-agent: Radiation Retriever 1.1
Disallow: /
User-agent: RepoMonkey
Disallow: /
User-agent: RepoMonkey Bait & Tackle/v1.01
Disallow: /
User-agent: Riddler
Disallow: /
User-agent: RMA
Disallow: /
User-agent: rojerbot
Disallow: /
User-agent: RyteBot
Disallow: /
User-agent: scooter
Disallow: /
User-agent: ScoutJet
Disallow: /
User-agent: Scrapy
Disallow: /
User-agent: ScreenerBot
Disallow: /
User-agent: searchmetrics
Disallow: /
User-agent: searchpreview
Disallow: /
User-agent: SemrushBot
Disallow: /
User-agent: sentibot
Disallow: /
User-agent: SEO-CRAWLING
Disallow: /
User-agent: SEOENGWorldBot
Disallow: /
User-agent: SEOkicks-Robot
Disallow: /
User-agent: ShopWiki
Disallow: /
User-agent: sistrix
Disallow: /
User-agent: sitebot
Disallow: /
User-agent: SiteSnagger
Disallow: /
User-agent: Snoopy
Disallow: /
User-agent: SocialSearcher
Disallow: /
User-agent: sogou
Disallow: /
User-agent: SolomonoBot
Disallow: /
User-agent: sootle
Disallow: /
User-agent: Sosospider
Disallow: /
User-agent: SpankBot
Disallow: /
User-agent: spanner
Disallow: /
User-agent: spbot
Disallow: /
User-agent: Speedy
Disallow: /
User-agent: Stanford
Disallow: /
User-agent: Stanford Comp Sci
Disallow: /
User-agent: SurveyBot
Disallow: /
User-agent: suzuran
Disallow: /
User-agent: Synapse
Disallow: /
User-agent: Szukacz/1.4
Disallow: /
User-agent: Teleport
Disallow: /
User-agent: TeleportPro
Disallow: /
User-agent: Telesoft
Disallow: /
User-agent: Teoma
Disallow: /
User-agent: The Intraformant
Disallow: /
User-agent: The\ Incutio\ XML-RPC\ PHP\ Library
Disallow: /
User-agent: TheNomad
Disallow: /
User-agent: toCrawl/UrlDispatcher
Disallow: /
User-agent: True_Robot
Disallow: /
User-agent: True_Robot/1.0
Disallow: /
User-agent: turingos
Disallow: /
User-agent: TurnitinBot
Disallow: /
User-agent: uCrawler
Disallow: /
User-agent: URL Control
Disallow: /
User-agent: URL_Spider_Pro
Disallow: /
User-agent: URLy Warning
Disallow: /
User-agent: VCI
Disallow: /
User-agent: VCI WebViewer VCI WebViewer Win32
Disallow: /
User-agent: visaduhoc\.info
Disallow: /
User-agent: WBSearchBot
Disallow: /
User-agent: Web Image Collector
Disallow: /
User-agent: WebAuto
Disallow: /
User-agent: WebBandit
Disallow: /
User-agent: WebBandit/3.50
Disallow: /
User-agent: WebCapture
Disallow: /
User-agent: WebCopier
Disallow: /
User-agent: WebEnhancer
Disallow: /
User-agent: WebInDetail\.com
Disallow: /
User-agent: WebmasterWorld Extractor
Disallow: /
User-agent: WebmasterWorldForumBot
Disallow: /
User-agent: WebSauger
Disallow: /
User-agent: Website Quester
Disallow: /
User-agent: WEBSITEtheWEB\.COM
Disallow: /
User-agent: Webster Pro
Disallow: /
User-agent: WebStripper
Disallow: /
User-agent: WebVac
Disallow: /
User-agent: WebZip
Disallow: /
User-agent: WebZip/4.0
Disallow: /
User-agent: Wget
Disallow: /
User-agent: Wget/1.5.3
Disallow: /
User-agent: Wget/1.6
Disallow: /
User-agent: Wotbot
Disallow: /
User-agent: http://www.trendiction.de/bot
Disallow: /
User-agent: www\.integromedb\.org
Disallow: /
User-agent: WWW-Collector-E
Disallow: /
User-agent: Xenu's
Disallow: /
User-agent: Xenu's Link Sleuth 1.1c
Disallow: /
User-agent: xpymep\.exe
Disallow: /
User-agent: YamanaLab-Robot
Disallow: /
User-agent: Yandex
Disallow: /
User-agent: YisouSpider
Disallow: /
User-agent: YodaoBot
Disallow: /
User-agent: YoudaoBot
Disallow: /
User-agent: Zend_Http_Client
Disallow: /
User-agent: Zeus
Disallow: /
User-agent: Zeus 32297 Webster Pro V2.9 Win32
Disallow: /
User-agent: Zeus Link Scout
Disallow: /
User-agent: ZmEu
Disallow: /
# END 2023-10-30
nginx