Найден корректный robots.txt
Файл управления сканированием сайта роботами
?
Robots.txt указывает поисковым роботам, какие страницы сканировать, а какие — нет. Ошибки в файле могут случайно закрыть важные разделы от индексации.
Robots.txt настроен корректно. Размер файла: 9076 байт. Загружен за: 2сек.
Проверяемая страница не запрещена в robots.txt.
Цепочка редиректов для файла robots.txt:
http://craigslist.ca/robots.txt
301 MovedPermanently
https://geo.craigslist.org/iso/ca
200 OK
Показать содержимое robots.txt
<!DOCTYPE html>
<html class="no-js">
<head>
<title>craigslist | canada</title>
<link type="text/css" rel="stylesheet" media="all" href="//www.craigslist.org/styles/simple-page.css?v=c2bdb0d98e47c9e35505a47872b8b53d">
<link type="text/css" rel="stylesheet" media="all" href="//www.craigslist.org/styles/jquery-ui-clcustom.css?v=76cc2a9e225c9050390ddd05f68f3560">
<link type="text/css" rel="stylesheet" media="all" href="//www.craigslist.org/styles/jquery.qtip-2.2.1.css?v=cd202aead4d1dd4894fbae4ade23fcf8">
<meta name="viewport" content="width=device-width,initial-scale=1">
</head>
<body class="simple geo footer header">
<script>
(function () {
var htmlElement = document.documentElement;
htmlElement.classList.remove('no-js');
htmlElement.classList.add('js');
var pageMode = window.pagemode = screen.width <= 480 ? 'mobile' : 'desktop';
document.body.classList.add(pageMode);
if ((window.innerWidth || document.documentElement.clientWidth) > 1000) {
document.body.classList.add('w1024');
}
})();
</script>
<div class="page-container flexed-footer">
<header class="global-header simple">
<a class="header-logo" name="logoLink" href="https://www.craigslist.org">CL</a>
<nav class="breadcrumbs-container">
<ul class="breadcrumbs">
<li class="crumb">
<p>
canada
</p>
</li>
</ul>
</nav>
</header>
<section class="simple-page-content">
<div class="geo-site-list-container">
<h1>choose the site nearest you:</h1>
<ul class="height6 geo-site-list">
<li><a href="https://www.craigslist.org/area/barrie">barrie, ON</a></li>
<li><a href="https://www.craigslist.org/area/belleville">belleville, ON</a></li>
<li><a href="https://www.craigslist.org/area/brantford">brantford-woodstock</a></li>
<li><a href="https://www.craigslist.org/area/calgary">calgary, AB</a></li>
<li><a href="https://www.craigslist.org/area/cariboo">cariboo, BC</a></li>
<li><a href="https://www.craigslist.org/area/chatham">chatham-kent, ON</a></li>
<li><a href="https://www.craigslist.org/area/comoxvalley">comox valley, BC</a></li>
<li><a href="https://www.craigslist.org/area/cornwall">cornwall, ON</a></li>
<li><a href="https://www.craigslist.org/area/edmonton">edmonton, AB</a></li>
<li><a href="https://www.craigslist.org/area/abbotsford">fraser valley, BC</a></li>
<li><a href="https://www.craigslist.org/area/ftmcmurray">ft mcmurray, AB</a></li>
<li><a href="https://www.craigslist.org/area/guelph">guelph, ON</a></li>
<li><a href="https://www.craigslist.org/area/halifax">halifax, NS</a></li>
<li><a href="https://www.craigslist.org/area/hamilton">hamilton-burlington</a></li>
<li><a href="https://www.craigslist.org/area/kamloops">kamloops, BC</a></li>
<li><a href="https://www.craigslist.org/area/kelowna">kelowna / okanagan</a></li>
<li><a href="https://www.craigslist.org/area/kingston">kingston, ON</a></li>
<li><a href="https://www.craigslist.org/area/kitchener">kitchener-waterloo-cambridge</a></li>
<li><a href="https://www.craigslist.org/area/kootenays">kootenays, BC</a></li>
<li><a href="https://www.craigslist.org/area/lethbridge">lethbridge, AB</a></li>
<li><a href="https://www.craigslist.org/area/londonon">london, ON</a></li>
<li><a href="https://www.craigslist.org/area/hat">medicine hat, AB</a></li>
<li><a href="https://www.craigslist.org/area/montreal"><b>montreal, QC</b></a></li>
<li><a href="https://www.craigslist.org/area/nanaimo">nanaimo, BC</a></li>
<li><a href="https://www.craigslist.org/area/newbrunswick">new brunswick</a></li>
<li><a href="https://www.craigslist.org/area/niagara">niagara region</a></li>
<li><a href="https://www.craigslist.org/area/ottawa">ottawa-hull-gatineau</a></li>
<li><a href="https://www.craigslist.org/area/owensound">owen sound, ON</a></li>
<li><a href="https://www.craigslist.org/area/peace">peace river country</a></li>
<li><a href="https://www.craigslist.org/area/peterborough">peterborough, ON</a></li>
<li><a href="https://www.craigslist.org/area/pei">prince edward island</a></li>
<li><a href="https://www.craigslist.org/area/princegeorge">prince george, BC</a></li>
<li><a href="https://www.craigslist.org/area/quebec">quebec city</a></li>
<li><a href="https://www.craigslist.org/area/reddeer">red deer, AB</a></li>
<li><a href="https://www.craigslist.org/area/regina">regina, SK</a></li>
<li><a href="https://www.craigslist.org/area/saguenay">saguenay, QC</a></li>
<li><a href="https://www.craigslist.org/area/sarnia">sarnia, ON</a></li>
<li><a href="https://www.craigslist.org/area/saskatoon">saskatoon, SK</a></li>
<li><a href="https://www.craigslist.org/area/soo">sault ste marie, ON</a></li>
<li><a href="https://www.craigslist.org/area/sherbrooke">sherbrooke, QC</a></li>
<li><a href="https://www.craigslist.org/area/skeena">skeena-bulkley</a></li>
<li><a href="https://www.craigslist.org/area/newfoundland">st john's, NL</a></li>
<li><a href="https://www.craigslist.org/area/sudbury">sudbury, ON</a></li>
<li><a href="https://www.craigslist.org/area/sunshine">sunshine coast, BC</a></li>
<li><a href="https://www.craigslist.org/area/territories">territories</a></li>
<li><a href="https://www.craigslist.org/area/thunderbay">thunder bay, ON</a></li>
<li><a href="https://www.craigslist.org/area/toronto"><b>toronto</b></a></li>
<li><a href="https://www.craigslist.org/area/troisrivieres">trois-rivieres, QC</a></li>
<li><a href="https://www.craigslist.org/area/vancouver"><b>vancouver, BC</b></a></li>
<li><a href="https://www.craigslist.org/area/victoria">victoria, BC</a></li>
<li><a href="https://www.craigslist.org/area/whistler">whistler / squamish</a></li>
<li><a href="https://www.craigslist.org/area/whitehorse">whitehorse, YT</a></li>
<li><a href="https://www.craigslist.org/area/windsor">windsor, ON</a></li>
<li><a href="https://www.craigslist.org/area/winnipeg">winnipeg, MB</a></li>
<li><a href="https://www.craigslist.org/area/yellowknife">yellowknife, NT</a></li>
</ul>
<div>
</section>
<div class="cl-footer-tx-spacer"></div>
<footer class="cl-footer-tx">
<div class="link-container">
<a href="https://www.craigslist.org/about/help/">help</a>
<a href="https://www.craigslist.org/about/help/safety">safety</a>
<a href="https://www.craigslist.org/about/privacy.policy">privacy</a>
<a href="https://www.craigslist.org/about/terms.of.use">terms</a>
<div class="link-container-nowrap">
<a href="https://www.craigslist.org/about/">about</a>
<a href="https://www.craigslist.org/about/craigslist_app">app</a>
</div>
</div>
<div class="copyright">© 2026 craigslist</div>
</footer>
</div>
<script src="//www.craigslist.org/js/jquery-3.7.1.min.js?v=0689db1d7619cfdb9fb02b72582e56b3" ></script>
<script src="//www.craigslist.org/js/jquery-migrate-3.5.2.min.js?v=3513cd8362227b3a09d52ddae90722fd" ></script>
<script>
(function(){
var cl = window.CL = window.cl = window.cl || window.CL || {};
cl.sandbox = "";
})();
try {
var p = document.createElement("p");
if (
!/\/\/.+\.craigslist\.org\/about\//.test(window.location.href) &&
window.location.pathname.length > 1 &&
(!window.addEventListener || JSON.parse(JSON.stringify('x')) !== 'x' || (p.style.flex!=='' && p.style['-webkit-flex'] !== ''))
) {
throw "unsupported browser";
}
} catch (e) {
function cleanup() {
document.body.innerHTML = '<div id="cl-unsupported-browser" style="margin:1em;font-size:150%;text-align:center;">We have detected you are using a browser that is missing critical features.<br><br>Please visit craigslist from a modern browser.</div>';
document.body.style = "display:block";
}
try {
document.body.style = "display:none";
} catch (e) {
}
window.onload = cleanup;
window.clUnsupportedBrowser = true;
}
</script>
<script src="//www.craigslist.org/js/general-concat.min.js?v=3bff0a73a12f1f4b91d159ae00b78c96" ></script>
</body>
</html>
