Найден корректный robots.txt
Файл управления сканированием сайта роботами
?
Robots.txt указывает поисковым роботам, какие страницы сканировать, а какие — нет. Ошибки в файле могут случайно закрыть важные разделы от индексации.
Кол-во редиректов для файла robots.txt 3 слишком большое! Это может вызывать затруднение при индексации поисковыми роботами!
Проверяемая страница не запрещена в robots.txt.
Цепочка редиректов для файла robots.txt:
http://wiki.xda-developers.com/robots.txt
301 MovedPermanently
https://wiki.xda-developers.com/robots.txt
302 Found
https://wiki.xda-developers.com/doku.php
200 OK
Показать содержимое robots.txt
<!DOCTYPE html>
<html lang="en" dir="ltr" class="no-js">
<head>
<meta charset="utf-8" />
<title>start [XDA Wiki]</title>
<script>(function(H){H.className=H.className.replace(/\bno-js\b/,'js')})(document.documentElement)</script>
<meta name="generator" content="DokuWiki"/>
<meta name="theme-color" content="#008800"/>
<meta name="robots" content="noindex,nofollow"/>
<link rel="search" type="application/opensearchdescription+xml" href="/lib/exe/opensearch.php" title="XDA Wiki"/>
<link rel="start" href="/"/>
<link rel="contents" href="/doku.php?id=start&do=index" title="Sitemap"/>
<link rel="manifest" href="/lib/exe/manifest.php"/>
<link rel="alternate" type="application/rss+xml" title="Recent Changes" href="/feed.php"/>
<link rel="alternate" type="application/rss+xml" title="Current namespace" href="/feed.php?mode=list&ns="/>
<link rel="alternate" type="text/html" title="Plain HTML" href="/doku.php?do=export_xhtml&id=start"/>
<link rel="alternate" type="text/plain" title="Wiki Markup" href="/doku.php?do=export_raw&id=start"/>
<link rel="stylesheet" href="/lib/exe/css.php?t=dokuwiki&tseed=ada985f147cc550884a166037502b663"/>
<!--[if gte IE 9]><!-->
<script >/*<![CDATA[*/var NS='';var JSINFO = {"id":"start","namespace":"","ACT":"denied","useHeadingNavigation":0,"useHeadingContent":0};
/*!]]>*/</script>
<script charset="utf-8" src="/lib/exe/jquery.php?tseed=34a552433bc33cc9c3bc32527289a0b2" defer="defer"></script>
<script charset="utf-8" src="/lib/exe/js.php?t=dokuwiki&tseed=ada985f147cc550884a166037502b663" defer="defer"></script>
<!--<![endif]-->
<meta name="viewport" content="width=device-width,initial-scale=1" />
<link rel="shortcut icon" href="/lib/tpl/dokuwiki/images/favicon.ico" />
<link rel="apple-touch-icon" href="/lib/tpl/dokuwiki/images/apple-touch-icon.png" />
</head>
<body>
<div id="dokuwiki__site"><div id="dokuwiki__top" class="site dokuwiki mode_denied tpl_dokuwiki notFound home ">
<!-- ********** HEADER ********** -->
<div id="dokuwiki__header"><div class="pad group">
<div class="headings group">
<ul class="a11y skip">
<li><a href="#dokuwiki__content">skip to content</a></li>
</ul>
<h1><a href="/doku.php?id=start" accesskey="h" title="[H]"><img src="/lib/tpl/dokuwiki/images/logo.png" width="64" height="64" alt="" /> <span>XDA Wiki</span></a></h1>
</div>
<div class="tools group">
<!-- USER TOOLS -->
<div id="dokuwiki__usertools">
<h3 class="a11y">User Tools</h3>
<ul>
<li class="action login"><a href="/doku.php?id=start&do=login&sectok=" title="Log In" rel="nofollow"><span>Log In</span><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path d="M10 17.25V14H3v-4h7V6.75L15.25 12 10 17.25M8 2h9a2 2 0 0 1 2 2v16a2 2 0 0 1-2 2H8a2 2 0 0 1-2-2v-4h2v4h9V4H8v4H6V4a2 2 0 0 1 2-2z"/></svg></a></li> </ul>
</div>
<!-- SITE TOOLS -->
<div id="dokuwiki__sitetools">
<h3 class="a11y">Site Tools</h3>
<form action="/doku.php?id=start" method="get" role="search" class="search doku_form" id="dw__search" accept-charset="utf-8"><input type="hidden" name="do" value="search" /><input type="hidden" name="id" value="start" /><div class="no"><input name="q" type="text" class="edit" title="[F]" accesskey="f" placeholder="Search" autocomplete="on" id="qsearch__in" value="" /><button value="1" type="submit" title="Search">Search</button><div id="qsearch__out" class="ajax_qsearch JSpopup"></div></div></form> <div class="mobileTools">
<form action="/doku.php" method="get" accept-charset="utf-8"><div class="no"><input type="hidden" name="id" value="start" /><select name="do" class="edit quickselect" title="Tools"><option value="">Tools</option><optgroup label="Page Tools"><option value="">Show page</option><option value="revisions">Old revisions</option><option value="backlink">Backlinks</option></optgroup><optgroup label="Site Tools"><option value="recent">Recent Changes</option><option value="media">Media Manager</option><option value="index">Sitemap</option></optgroup><optgroup label="User Tools"><option value="login">Log In</option></optgroup></select><button type="submit">></button></div></form> </div>
<ul>
<li class="action recent"><a href="/doku.php?id=start&do=recent" title="Recent Changes [r]" rel="nofollow" accesskey="r">Recent Changes</a></li><li class="action media"><a href="/doku.php?id=start&do=media&ns=" title="Media Manager" rel="nofollow">Media Manager</a></li><li class="action index"><a href="/doku.php?id=start&do=index" title="Sitemap [x]" accesskey="x">Sitemap</a></li> </ul>
</div>
</div>
<!-- BREADCRUMBS -->
<div class="breadcrumbs">
<div class="trace"><span class="bchead">Trace:</span></div>
</div>
<hr class="a11y" />
</div></div><!-- /header -->
<div class="wrapper group">
<!-- ********** CONTENT ********** -->
<div id="dokuwiki__content"><div class="pad group">
<div class="pageId"><span>start</span></div>
<div class="page group">
<!-- wikipage start -->
<h1 class="sectionedit1" id="permission_denied">Permission Denied</h1>
<div class="level1">
<p>
Sorry, you don't have enough rights to continue.
</p>
</div>
<h1 class="sectionedit1" id="login">Login</h1>
<div class="level1">
<p>
You are currently not logged in! Enter your authentication credentials below to log in. You need to have cookies enabled to log in.
</p>
</div>
<div class="centeralign">
<form id="dw__login" action="/doku.php?id=start" method="post" accept-charset="utf-8"><div class="no">
<input type="hidden" name="sectok" value="" /><input type="hidden" name="id" value="start" /><input type="hidden" name="do" value="login" /><fieldset ><legend>Log In</legend>
<label class="block" for="focus__this"><span>Username</span> <input type="text" id="focus__this" name="u" class="edit" /></label><br />
<label class="block"><span>Password</span> <input type="password" name="p" class="edit" /></label><br />
<label class="simple" for="remember__me"><input type="checkbox" id="remember__me" name="r" value="1" /> <span>Remember me</span></label>
<button type="submit">Log In</button>
</fieldset>
</div></form>
</div>
<!-- wikipage stop -->
</div>
<div class="docInfo"></div>
</div></div><!-- /content -->
<hr class="a11y" />
<!-- PAGE ACTIONS -->
<div id="dokuwiki__pagetools">
<h3 class="a11y">Page Tools</h3>
<div class="tools">
<ul>
<li class="show"><a href="/doku.php?id=start&do=" title="Show page [v]" rel="nofollow" accesskey="v"><span>Show page</span><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path d="M13 9h5.5L13 3.5V9M6 2h8l6 6v12a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V4c0-1.11.89-2 2-2m9 16v-2H6v2h9m3-4v-2H6v2h12z"/></svg></a></li><li class="revs"><a href="/doku.php?id=start&do=revisions" title="Old revisions [o]" rel="nofollow" accesskey="o"><span>Old revisions</span><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path d="M11 7v5.11l4.71 2.79.79-1.28-4-2.37V7m0-5C8.97 2 5.91 3.92 4.27 6.77L2 4.5V11h6.5L5.75 8.25C6.96 5.73 9.5 4 12.5 4a7.5 7.5 0 0 1 7.5 7.5 7.5 7.5 0 0 1-7.5 7.5c-3.27 0-6.03-2.09-7.06-5h-2.1c1.1 4.03 4.77 7 9.16 7 5.24 0 9.5-4.25 9.5-9.5A9.5 9.5 0 0 0 12.5 2z"/></svg></a></li><li class="backlink"><a href="/doku.php?id=start&do=backlink" title="Backlinks" rel="nofollow"><span>Backlinks</span><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path d="M10.59 13.41c.41.39.41 1.03 0 1.42-.39.39-1.03.39-1.42 0a5.003 5.003 0 0 1 0-7.07l3.54-3.54a5.003 5.003 0 0 1 7.07 0 5.003 5.003 0 0 1 0 7.07l-1.49 1.49c.01-.82-.12-1.64-.4-2.42l.47-.48a2.982 2.982 0 0 0 0-4.24 2.982 2.982 0 0 0-4.24 0l-3.53 3.53a2.982 2.982 0 0 0 0 4.24m2.82-4.24c.39-.39 1.03-.39 1.42 0a5.003 5.003 0 0 1 0 7.07l-3.54 3.54a5.003 5.003 0 0 1-7.07 0 5.003 5.003 0 0 1 0-7.07l1.49-1.49c-.01.82.12 1.64.4 2.43l-.47.47a2.982 2.982 0 0 0 0 4.24 2.982 2.982 0 0 0 4.24 0l3.53-3.53a2.982 2.982 0 0 0 0-4.24.973.973 0 0 1 0-1.42z"/></svg></a></li><li class="top"><a href="#dokuwiki__top" title="Back to top [t]" rel="nofollow" accesskey="t"><span>Back to top</span><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8v12z"/></svg></a></li> </ul>
</div>
</div>
</div><!-- /wrapper -->
<!-- ********** FOOTER ********** -->
<div id="dokuwiki__footer"><div class="pad">
<div class="buttons">
<a href="https://www.dokuwiki.org/donate" title="Donate" ><img
src="/lib/tpl/dokuwiki/images/button-donate.gif" width="80" height="15" alt="Donate" /></a>
<a href="https://php.net" title="Powered by PHP" ><img
src="/lib/tpl/dokuwiki/images/button-php.gif" width="80" height="15" alt="Powered by PHP" /></a>
<a href="//validator.w3.org/check/referer" title="Valid HTML5" ><img
src="/lib/tpl/dokuwiki/images/button-html5.png" width="80" height="15" alt="Valid HTML5" /></a>
<a href="//jigsaw.w3.org/css-validator/check/referer?profile=css3" title="Valid CSS" ><img
src="/lib/tpl/dokuwiki/images/button-css.png" width="80" height="15" alt="Valid CSS" /></a>
<a href="https://dokuwiki.org/" title="Driven by DokuWiki" ><img
src="/lib/tpl/dokuwiki/images/button-dw.png" width="80" height="15"
alt="Driven by DokuWiki" /></a>
</div>
</div></div><!-- /footer -->
</div></div><!-- /site -->
<div class="no"><img src="/lib/exe/taskrunner.php?id=start&1787565634" width="2" height="1" alt="" /></div>
<div id="screen__mode" class="no"></div></body>
</html>
nginx