Найден корректный robots.txt
Файл управления сканированием сайта роботами
?
Robots.txt указывает поисковым роботам, какие страницы сканировать, а какие — нет. Ошибки в файле могут случайно закрыть важные разделы от индексации.
Кол-во редиректов для файла robots.txt 3 слишком большое! Это может вызывать затруднение при индексации поисковыми роботами!
Проверяемая страница не запрещена в robots.txt.
Цепочка редиректов для файла robots.txt:
http://thisismyengine.com/robots.txt
301 MovedPermanently
https://thisismyengine.com/robots.txt
307 RedirectKeepVerb
https://beta.thisismyengine.com/
200 OK
Показать содержимое robots.txt
<!-- top -->
<!DOCTYPE html>
<html class="env:develop view:index browser:load" lang="en">
<head>
<!-- encoding -->
<meta charset="utf-8">
<!-- viewport -->
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0">
<!-- title -->
<title>Engine — Digital Scientists</title>
<!-- description -->
<meta name="description" content="We collaborate with leading brands and agencies to produce carefully crafted digital tools and experiences.">
<!-- favicon -->
<link href="https://beta.thisismyengine.com/src/gfx/favicon.png" sizes="192x192" rel="icon">
<!-- style : site -->
<link href="https://beta.thisismyengine.com/src/css/site.css" rel="stylesheet">
<!-- plugin : jquery -->
<script src="https://beta.thisismyengine.com/src/plugin/jquery/jquery.js"></script>
<!-- script : site -->
<script src="https://beta.thisismyengine.com/src/script/site.js"></script>
<!-- top : head -->
<!-- share : og -->
<meta property="og:type" content="website">
<meta property="og:url" content="https://beta.thisismyengine.com/">
<meta property="og:title" content="Engine — Digital Scientists">
<meta property="og:description" content="We collaborate with leading brands and agencies to produce carefully crafted digital tools and experiences.">
<meta property="og:image" content="https://beta.thisismyengine.com/src/gfx/share.png">
<!-- share : twitter -->
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:url" content="https://beta.thisismyengine.com/">
<meta name="twitter:title" content="Engine — Digital Scientists">
<meta name="twitter:description" content="We collaborate with leading brands and agencies to produce carefully crafted digital tools and experiences.">
<meta name="twitter:image" content="https://beta.thisismyengine.com/src/gfx/share.png">
<!-- ROOT -->
<script>var ROOT_URL = 'https://beta.thisismyengine.com';</script>
</head>
<body>
<!-- header -->
<header>
<nav class="primary">
<a href="https://beta.thisismyengine.com">Engine</a>
</nav>
<nav class="secondary">
<a href="https://beta.thisismyengine.com/collaborators">Collaborators</a>
<a href="https://beta.thisismyengine.com/studio">Studio</a>
<a href="https://beta.thisismyengine.com/principles">Principles</a>
<a href="https://beta.thisismyengine.com/journal">Journal</a>
<a href="https://beta.thisismyengine.com/listen">Listen</a>
</nav>
</nav>
<button class="toggle" type="button"></button>
</header>
<!-- main -->
<main>
<!-- intro -->
<section class="intro">
<div class="text">
<p>We collaborate with leading brands and agencies to produce carefully crafted digital tools and experiences.</p>
</div>
</section>
<!-- feature -->
<section class="feature">
<div class="grid">
<article class="feature feature:project" style="--color-hex: #f57d52; --color-dp3: color(display-p3 0.96 0.49 0.32)" data-feature-id="19">
<a href="https://beta.thisismyengine.com/blond-project" target="_self">
<div class="image">
<picture>
<img src="https://beta.thisismyengine.com/lib/feature/19_list_4.webp">
</picture>
</div>
<div class="content">
<div class="subject">Project</div>
<div class="name">Blond</div>
</div>
</a>
</article>
<article class="feature feature:project" style="--color-hex: #d7fa05; --color-dp3: color(display-p3 0.84 0.98 0.02)" data-feature-id="18">
<a href="https://beta.thisismyengine.com/made-lab" target="_self">
<div class="image">
<picture>
<img src="https://beta.thisismyengine.com/lib/feature/18_list_4.webp">
</picture>
</div>
<div class="content">
<div class="subject">Project</div>
<div class="name">Made[LAB]</div>
</div>
</a>
</article>
<article class="feature feature:interview" style="--color-hex: #d09b4f; --color-dp3: color(display-p3 0.82 0.61 0.31)" data-feature-id="20">
<a href="https://beta.thisismyengine.com/ai-as-infrastructure-rob-boyett" target="_self">
<div class="image">
<picture>
<img src="https://beta.thisismyengine.com/lib/feature/20_list_4.webp">
</picture>
</div>
<div class="content">
<div class="subject">Interview</div>
<div class="name">Interview with Rob Boyett (Pt I)</div>
</div>
</a>
</article>
<article class="feature feature:project" style="--color-hex: #ddb891; --color-dp3: color(display-p3 0.87 0.72 0.57)" data-feature-id="8">
<a href="https://beta.thisismyengine.com/maggs-project" target="_self">
<div class="image">
<picture>
<img src="https://beta.thisismyengine.com/lib/feature/8_list_4.webp">
</picture>
</div>
<div class="content">
<div class="subject">Project</div>
<div class="name">Maggs</div>
</div>
</a>
</article>
<article class="feature feature:project" style="--color-hex: #e6ff9d; --color-dp3: color(display-p3 0.9 1 0.62)" data-feature-id="17">
<a href="https://beta.thisismyengine.com/webb" target="_self">
<div class="image">
<picture>
<img src="https://beta.thisismyengine.com/lib/feature/17_list_4.webp">
</picture>
</div>
<div class="content">
<div class="subject">Project</div>
<div class="name">Webb</div>
</div>
</a>
</article>
<article class="feature feature:project" style="--color-hex: #f57a53; --color-dp3: color(display-p3 0.96 0.48 0.33)" data-feature-id="2">
<a href="https://beta.thisismyengine.com/paul-archer-design-project" target="_self">
<div class="image">
<picture>
<img src="https://beta.thisismyengine.com/lib/feature/2_list_4.webp">
</picture>
</div>
<div class="content">
<div class="subject">Project</div>
<div class="name">Paul Archer Design</div>
</div>
</a>
</article>
<article class="feature feature:project" style="--color-hex: #eeeee4; --color-dp3: color(display-p3 0.93 0.93 0.89)" data-feature-id="15">
<a href="https://beta.thisismyengine.com/bcmh-website" target="_self">
<div class="image">
<picture>
<img src="https://beta.thisismyengine.com/lib/feature/15_list_4.webp">
</picture>
</div>
<div class="content">
<div class="subject">Project</div>
<div class="name">BCMH Website</div>
</div>
</a>
</article>
<article class="feature feature:space" style="--color-hex: #2d2d4f; --color-dp3: color(display-p3 0.18 0.18 0.31)" data-feature-id="4">
<a href="https://beta.thisismyengine.com/hoare-associates-spaces" target="_self">
<div class="image">
<picture>
<img src="https://beta.thisismyengine.com/lib/feature/4_list_4.webp">
</picture>
</div>
<div class="content">
<div class="subject">Spaces</div>
<div class="name">Hoare Associates Spaces</div>
</div>
</a>
</article>
<article class="feature feature:project" style="--color-hex: #ffffff; --color-dp3: color(display-p3 1 1 1)" data-feature-id="11">
<a href="https://beta.thisismyengine.com/aus-music-project" target="_self">
<div class="image">
<picture>
<img src="https://beta.thisismyengine.com/lib/feature/11_list_4.webp">
</picture>
</div>
<div class="content">
<div class="subject">Project</div>
<div class="name">Aus Music</div>
</div>
</a>
</article>
<article class="feature feature:project" style="--color-hex: #c3b071; --color-dp3: color(display-p3 0.76 0.69 0.44)" data-feature-id="7">
<a href="https://beta.thisismyengine.com/max-cooper-emergence-project" target="_self">
<div class="image">
<picture>
<img src="https://beta.thisismyengine.com/lib/feature/7_list_4.webp">
</picture>
</div>
<div class="content">
<div class="subject">Project</div>
<div class="name">Emergence</div>
</div>
</a>
</article>
</div>
</section>
</main>
<!-- footer -->
<footer>
<a class="logo" href="https://beta.thisismyengine.com"></a>
<nav class="social">
<a class="instagram" href="https://www.instagram.com/thisismyengine" target="_blank">Instagram</a>
<a class="twitter" href="https://twitter.com/thisismyengine" target="_blank">Twitter</a>
<a class="facebook" href="https://www.facebook.com/thisismyengine" target="_blank">Facebook</a>
</nav>
<nav class="contact">
<a class="email" href="/cdn-cgi/l/email-protection#5038353c3c3f102438392339233d29353e37393e357e333f3d">
<span><span class="__cf_email__" data-cfemail="bed6dbd2d2d1fecad6d7cdd7cdd3c7dbd0d9d7d0db90ddd1d3">[email protected]</span></span>
</a>
<a class="phone" href="tel:+44(0)1373 800420">
<span>+44(0)1373 800420<span>
</a>
</nav>
</footer>
<!-- bottom -->
<script data-cfasync="false" src="/cdn-cgi/scripts/5c5dd728/cloudflare-static/email-decode.min.js"></script><script type="module" src="https://static.cloudflareinsights.com/beacon.min.js/v4513226cdae34746b4dedf0b4dfa099e1781791509496" integrity="sha512-ZE9pZaUXND66v380QUtch/5sE9tPFh2zg45pR2PB0CVkCtOREv2AJKkSidISWkysEuQ0EH8faUU5du78bx87UQ==" data-cf-beacon='{"version":"2024.11.0","token":"7d9fe869e0124d73bc825748edd88c90","r":1}' crossorigin="anonymous"></script>
</body>
</html>
Cloudflare