Найден корректный robots.txt
Файл управления сканированием сайта роботами
?
Robots.txt указывает поисковым роботам, какие страницы сканировать, а какие — нет. Ошибки в файле могут случайно закрыть важные разделы от индексации.
Кол-во редиректов для файла robots.txt 3 слишком большое! Это может вызывать затруднение при индексации поисковыми роботами!
Проверяемая страница не запрещена в robots.txt.
Цепочка редиректов для файла robots.txt:
http://livemaster.ru/robots.txt
301 MovedPermanently
https://livemaster.ru/robots.txt
301 MovedPermanently
https://www.livemaster.ru/robots.txt
200 OK
Показать содержимое robots.txt
Sitemap: https://www.livemaster.ru/sitemaps/ru/sitemapIndex.xml
Sitemap: https://www.livemaster.ru/sitemapsImages/ru/sitemapIndex.xml
LLMS: https://www.livemaster.ru/llms.txt
User-agent: *
# Technical and service sections
Disallow: /blitz/
Disallow: /direct/
Disallow: /direct$
Disallow: /article/job
Disallow: /itemgallery/
Disallow: /itemgallery$
Disallow: /payments/
Disallow: /payments$
Disallow: /order/
Disallow: /order$
Disallow: /questions/
Disallow: /questions$
Disallow: /advertising/
Disallow: /advertising$
Disallow: /auth/
Disallow: /auth$
Disallow: /request/
Disallow: /request$
Disallow: /cart/
Disallow: /cart$
Disallow: /pfs/
Disallow: /pfs$
Disallow: /extensions/
Disallow: /extensions$
Disallow: /api/*
Disallow: /extapi/
Disallow: /extapi$
# User accounts and private sections
Disallow: /myshop/
Disallow: /myshop$
Disallow: /purchase/
Disallow: /purchase$
Disallow: /purchases/
Disallow: /purchases$
Disallow: /sales/
Disallow: /sales$
Disallow: /myfavorite/
Disallow: /myfavorite$
Disallow: /mygallery/
Disallow: /mygallery$
Disallow: /myevents/
Disallow: /myevents$
Disallow: /usergallery/
Disallow: /usergallery$
Disallow: /followers/
Disallow: /followers$
Disallow: /following/
Disallow: /following$
Disallow: /contacts/
Disallow: /contacts$
Disallow: /*/contact
Disallow: /feed$
Disallow: /subscribes/
Disallow: /subscribes$
Disallow: /wishlist/
Disallow: /wishlist$
# Promo and temporary sections
Disallow: /bigsale/
Disallow: /bigsale$
Disallow: /foaf/
Disallow: /foaf$
Disallow: /perfectsoap$
Disallow: /perfectsoap/profile
Disallow: /referrallink/
Disallow: /referrallink$
Disallow: /handicraftday/
Disallow: /handicraftday$
Disallow: /statistics2020/
Disallow: /statistics2020$
Disallow: /saveprincess/
Disallow: /saveprincess$
Disallow: /getyourpassport/
Disallow: /getyourpassport$
Disallow: /events/
Disallow: /events$
Disallow: /blogs/
Disallow: /blogs$
Disallow: /forum/
Disallow: /forum$
Disallow: /promo/
Disallow: /promo$
Disallow: /popular/item/
# Legacy PHP endpoints
Disallow: /purchase.php
Disallow: /main.php
Disallow: /controller.php
Disallow: /user.php?id=
Disallow: /feedbacks?uid=
Disallow: *search.php
Disallow: /message.php
Disallow: /register.php
Disallow: /masterclasses.php
Disallow: /lastlist.php
Disallow: /favitems.php
Disallow: /item/edit
# Profile and policy URLs
Disallow: */followers
Disallow: */reviews
Disallow: */policy
# Tracking, sorting and dynamic parameters
Disallow: *sort=
Disallow: *view=
Disallow: /*utm_
Disallow: /*ysclid=
Disallow: *?action
Disallow: *?&search
Disallow: *?search
Disallow: *?pos
Disallow: *?&pos
Disallow: *?nlsi
Disallow: *?c
Disallow: *?fromfeed
Disallow: *?invcode
Disallow: *?vr
Disallow: *?inside
Disallow: *?msec
Disallow: *?sortitems
Disallow: *?wf
Disallow: *?userGallery
Disallow: *?v
Disallow: *?ctype
Disallow: *?viewTypePage
Disallow: *?objectID
Disallow: *?status
Disallow: *?cat
Disallow: *?viewprofile
Disallow: *?testcomments
Disallow: *?cid
Disallow: *?ptypeday
Disallow: *?article
Disallow: *?content
Disallow: *?sorder
Disallow: *?sort
Disallow: *?tiaa
Disallow: *?s
Disallow: *?view
Disallow: *?id
Disallow: *?jid
Disallow: *?fromcomments
Disallow: *?maxp
Disallow: *?&tag
Disallow: *?showreg
Disallow: *?showShop
Disallow: /*sectiontype=
Disallow: /*bankcard=
Disallow: /*privilege=
Disallow: /*haveDiscount=
Disallow: /*exclusive=
Disallow: /*colors=
Disallow: /*materials=
Disallow: /*cityStr=
Disallow: /*sortitems=
Disallow: /*minprice=
Disallow: /*maxprice=
Disallow: /*block=
Disallow: /*element=
User-agent: Yandex
# Technical and service sections
Disallow: /direct/
Disallow: /direct$
Disallow: /blitz/
Disallow: /article/job
Disallow: /itemgallery/
Disallow: /itemgallery$
Disallow: /payments/
Disallow: /payments$
Disallow: /order/
Disallow: /order$
Disallow: /questions/
Disallow: /questions$
Disallow: /advertising/
Disallow: /advertising$
Disallow: /auth/
Disallow: /auth$
Disallow: /request/
Disallow: /request$
Disallow: /cart/
Disallow: /cart$
Disallow: /pfs/
Disallow: /pfs$
Disallow: /extensions/
Disallow: /extensions$
Disallow: /api/*
Disallow: /extapi/
Disallow: /extapi$
# User accounts and private sections
Disallow: /myshop/
Disallow: /myshop$
Disallow: /purchase/
Disallow: /purchase$
Disallow: /purchases/
Disallow: /purchases$
Disallow: /sales/
Disallow: /sales$
Disallow: /myfavorite/
Disallow: /myfavorite$
Disallow: /mygallery/
Disallow: /mygallery$
Disallow: /myevents/
Disallow: /myevents$
Disallow: /usergallery/
Disallow: /usergallery$
Disallow: /followers/
Disallow: /followers$
Disallow: /following/
Disallow: /following$
Disallow: /contacts/
Disallow: /contacts$
Disallow: /*/contact
Disallow: /feed$
Disallow: /subscribes/
Disallow: /subscribes$
Disallow: /wishlist/
Disallow: /wishlist$
# Promo and temporary sections
Disallow: /bigsale/
Disallow: /bigsale$
Disallow: /foaf/
Disallow: /foaf$
Disallow: /perfectsoap$
Disallow: /perfectsoap/profile
Disallow: /referrallink/
Disallow: /referrallink$
Disallow: /handicraftday/
Disallow: /handicraftday$
Disallow: /statistics2020/
Disallow: /statistics2020$
Disallow: /saveprincess/
Disallow: /saveprincess$
Disallow: /getyourpassport/
Disallow: /getyourpassport$
Disallow: /events/
Disallow: /events$
Disallow: /blogs/
Disallow: /blogs$
Disallow: /forum/
Disallow: /forum$
Disallow: /promo/
Disallow: /promo$
Disallow: /popular/item/
# Legacy PHP endpoints
Disallow: /purchase.php
Disallow: /main.php
Disallow: /controller.php
Disallow: /user.php?id=
Disallow: /feedbacks?uid=
Disallow: *search.php
Disallow: /message.php
Disallow: /register.php
Disallow: /masterclasses.php
Disallow: /lastlist.php
Disallow: /favitems.php
Disallow: /item/edit
# Profile and policy URLs
Disallow: */followers
Disallow: */reviews
Disallow: */policy
# Tracking, sorting and dynamic parameters
Disallow: *sort=
Disallow: *view=
Disallow: /*utm_
Disallow: /*ysclid=
Disallow: *?action
Disallow: *?&search
Disallow: *?search
Disallow: *?pos
Disallow: *?&pos
Disallow: *?nlsi
Disallow: *?c
Disallow: *?fromfeed
Disallow: *?invcode
Disallow: *?vr
Disallow: *?inside
Disallow: *?msec
Disallow: *?sortitems
Disallow: *?wf
Disallow: *?userGallery
Disallow: *?v
Disallow: *?ctype
Disallow: *?viewTypePage
Disallow: *?objectID
Disallow: *?status
Disallow: *?cat
Disallow: *?viewprofile
Disallow: *?testcomments
Disallow: *?cid
Disallow: *?ptypeday
Disallow: *?article
Disallow: *?content
Disallow: *?sorder
Disallow: *?sort
Disallow: *?tiaa
Disallow: *?s
Disallow: *?view
Disallow: *?id
Disallow: *?jid
Disallow: *?fromcomments
Disallow: *?maxp
Disallow: *?&tag
Disallow: *?showreg
Disallow: *?showShop
Disallow: /*sectiontype=
Disallow: /*bankcard=
Disallow: /*privilege=
Disallow: /*haveDiscount=
Disallow: /*exclusive=
Disallow: /*colors=
Disallow: /*materials=
Disallow: /*cityStr=
Disallow: /*sortitems=
Disallow: /*minprice=
Disallow: /*maxprice=
Disallow: /*block=
Disallow: /*element=
Clean-param: utm_source&utm_medium&utm_campaign&utm_term&utm_content&ysclid&lsi&login&zoom&inside&minp&gcmes§iontype&bankcard&privilege&haveDiscount&exclusive&colors&materials&cityStr&sortitems&minprice&maxprice&from
User-agent: AhrefsBot
Disallow: /
User-agent: 360Spider
Disallow: /
User-agent: 8LEGS
Disallow: /
User-agent: betaBot
Disallow: /
User-agent: BLEXBot
Disallow: /
User-agent: BoardReader
Disallow: /
User-agent: BPImageWalker
Disallow: /
User-agent: CheckMarkNetwork
Disallow: /
User-agent: Clockwork Data Vault
Disallow: /
User-agent: coccoc
Disallow: /
User-agent: CRAZYWEBCRAWLER
Disallow: /
User-agent: Domain Re-Animator
Disallow: /
User-agent: DomainStatsBot
Disallow: /
User-agent: DotBot
Disallow: /
User-agent: dotbot
Disallow: /
User-agent: EasouSpider
Disallow: /
User-agent: Ezooms
Disallow: /
User-agent: Findxbot
Disallow: /
User-agent: GetIntent
Disallow: /
User-agent: Grammarly
Disallow: /
User-agent: Guardcrwlr
Disallow: /
User-agent: HubSpot Links Crawler
Disallow: /
User-agent: istellabot
Disallow: /
User-agent: KOCMOHABT
Disallow: /
User-agent: LinksCrawler
Disallow: /
User-agent: linkdexbot
Disallow: /
User-agent: LinkSpammer
Disallow: /
User-agent: LinkWalker
Disallow: /
User-agent: ltx71
Disallow: /
# Allow RSS for Mail.ru
User-agent: Mail.RU_Bot
Allow: /rss
User-agent: MJ12bot
Disallow: /
User-agent: NetTrack
Disallow: /
User-agent: okhttp
Disallow: /
User-agent: OpenLinkProfiler
Disallow: /
User-agent: PaperLiBot
Disallow: /
User-agent: Pingdom
Disallow: /
User-agent: proximic
Disallow: /
User-agent: Pu_iN
Disallow: /
User-agent: Pulsepoint
Disallow: /
User-agent: Qwantify
Disallow: /
User-agent: RankvalBot
Disallow: /
User-agent: SalesIntelligent
Disallow: /
User-agent: Scrapy
Disallow: /
User-agent: Screaming Frog SEO Spider
Disallow: /
User-agent: Searchie
Disallow: /
User-agent: SemrushBot
Disallow: /
User-agent: SemrushBot-SA
Disallow: /
User-agent: SEOkicks-Robot
Disallow: /
User-agent: SocialRankIOBot
Disallow: /
User-agent: Sogou
Disallow: /
User-agent: Sogou web spider
Disallow: /
User-agent: spbot
Disallow: /
User-agent: SurveyBot
Disallow: /
User-agent: Uptimebot
Disallow: /
User-agent: Vegi bot
Disallow: /
User-agent: VeriCiteCrawler
Disallow: /
User-agent: webmeup-crawler
Disallow: /
User-agent: weborama-fetcher
Disallow: /
User-agent: YisouSpider
Disallow: /
User-agent: YoudaoBot
Disallow: /
nginx