Найден корректный robots.txt
Файл управления сканированием сайта роботами
?
Robots.txt указывает поисковым роботам, какие страницы сканировать, а какие — нет. Ошибки в файле могут случайно закрыть важные разделы от индексации.
Кол-во редиректов для файла robots.txt 4 слишком большое! Это может вызывать затруднение при индексации поисковыми роботами!
Проверяемая страница не запрещена в robots.txt.
Цепочка редиректов для файла robots.txt:
http://rafahtoday.org/robots.txt
301 MovedPermanently
https://rafahtoday.org/robots.txt
301 MovedPermanently
https://www.rafahtoday.org/robots.txt
301 MovedPermanently
https://www.rafahtoday.org/
200 OK
Показать содержимое robots.txt
<!DOCTYPE html>
<html>
<head>
<title>Rafah's website</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
<meta name="keywords" content="Palestine diaries, Cartoons, political cartoons, homeless families, home demolitions, Jenin, deaths and wounded, snapshots, pictures from Palestine, about Rafah, Rafah Refugee Camp, Rafah Today, homeless children, children wounded, Middle East, mideast conflict, intifada">
<meta name="description" content="On this website, I present photos and reports about my home town. About our life, our community, the home demolitions, homeless families, the children in our camp... About the tragedies that happen here every day.">
</head>
<body bgcolor="#000066" topmargin="0">
<table width="759" border="0" cellspacing="0" cellpadding="0" height="321">
<tr>
<td colspan="2">
<h2 align="right"><b><font color="white"><img src="regimages/topbar.gif" width="750" height="50"></font></b></h2>
</td>
</tr>
<tr>
<td width="140" valign="top" height="340">
<p><img src="regimages/ps1.jpg" width="140" height="143"><br></p>
<table width="140" border="0" cellspacing="0" cellpadding="0" height="233">
<tr>
<td height="21" valign="top" bgcolor="#990000">
<div align="right">
<a href="news/todaymain.htm"><font face="Arial, Helvetica, sans-serif"><font size="2" face="Verdana, Arial, Helvetica, sans-serif" color="#990000"><b><font color="#FFFFFF" size="3">Rafah Today</font></b></font></font></a> <b><a href="news/todaymain.htm"><font color="#FFFFFF" face="Arial, Helvetica, sans-serif"><font size="2" face="Verdana, Arial, Helvetica, sans-serif"></font></font></a></b>
</div>
</td>
</tr>
<tr>
<td height="16" valign="top" bgcolor="#FFFFFF"><font size="2" face="Arial, Helvetica, sans-serif"><b><a href="gazanews/gazamain.htm"><font color="#666666">Gaza News</font></a> <font color="#990000"></font></b></font></td>
</tr>
<tr>
<td height="16" valign="top">
<a href="news/special.htm"><font face="Verdana, Arial, Helvetica, sans-serif" size="2"><b><font color="#CCCCCC">Special reports</font></b></font></a> <font color="#FFFFFF" face="Verdana, Arial, Helvetica, sans-serif" size="2"><a href="news/special.htm"><b></b></a><b></b></font>
</td>
</tr>
<tr>
<td height="8" valign="top"><b><a href="about/rafah.htm"><font face="Verdana, Arial, Helvetica, sans-serif" size="2"><font color="#CCCCCC">About Rafah</font></font></a> <font color="#FFFFFF" face="Verdana, Arial, Helvetica, sans-serif" size="2"></font></b></td>
</tr>
<tr>
<td height="2" valign="top"><b><a href="details/demolitions.htm"><font face="Verdana, Arial, Helvetica, sans-serif" size="2"><font color="#CCCCCC">Home demolitions</font></font></a></b></td>
</tr>
<tr>
<td height="2" valign="top"><b><a href="details/homeless.htm"><font face="Verdana, Arial, Helvetica, sans-serif" size="2"><font color="#CCCCCC">Homeless families</font></font></a></b></td>
</tr>
<tr>
<td height="16" valign="top"><b><a href="details/deaths.htm"><font face="Verdana, Arial, Helvetica, sans-serif" size="2"><font color="#CCCCCC">Deaths & wounded</font></font></a> <font color="#FFFFFF" face="Verdana, Arial, Helvetica, sans-serif" size="2"></font></b></td>
</tr>
<tr>
<td height="10" valign="top"><b><a href="about/snaps.htm"><font face="Verdana, Arial, Helvetica, sans-serif" size="2"><font color="#CCCCCC">Snapshots</font></font></a> <font color="#FFFFFF" face="Verdana, Arial, Helvetica, sans-serif" size="2"></font></b></td>
</tr>
<tr>
<td height="12" valign="top"><b><a href="about/links.htm"><font face="Verdana, Arial, Helvetica, sans-serif" size="2"><font color="#CCCCCC">Links</font></font></a> <font color="#FFFFFF" face="Verdana, Arial, Helvetica, sans-serif" size="2"></font></b></td>
</tr>
<tr>
<td height="14" valign="top"><b><a href="details/cartoons.htm"><font face="Verdana, Arial, Helvetica, sans-serif" size="2"><font color="#CCCCCC">Cartoons</font></font></a> <font color="#FFFFFF" face="Verdana, Arial, Helvetica, sans-serif" size="2"></font></b></td>
</tr>
<tr>
<td height="23" valign="top"><b><a href="about/messages1.htm"><font face="Verdana, Arial, Helvetica, sans-serif" size="2"><font color="#CCCCCC">Messages/Contact</font></font></a></b></td>
</tr>
<tr>
<td height="23" valign="top">
<div align="right">
<font size="2" face="Verdana, Arial, Helvetica, sans-serif" color="#CCCCCC">
<script language="JavaScript" src="http://server.scripthost.com/counter?rafah"></script></font>
</div>
</td>
</tr>
</table>
</td>
<td width="619" height="340" valign="top">
<table width="616" border="0" cellspacing="0" cellpadding="0" align="center" height="431">
<tr>
<td bgcolor="#FFFFFF" valign="top" height="401">
<div align="center">
<table width="570" border="0" cellspacing="0" cellpadding="5">
<tr>
<td width="450"><img src="regimages/rafah.jpg" width="450" height="368"></td>
<td width="100">
<font size="1" face="Verdana, Arial, Helvetica, sans-serif">Home demolitions in Rafah.<br>
Bulldozers erase densely populated areas,<br>
to make room for the new "iron wall".<br>
The families lose everything.</font> <font size="2" face="Verdana, Arial, Helvetica, sans-serif"></font>
</td>
</tr>
</table>
</div>
<blockquote>
<div align="left">
<font size="2" face="Verdana, Arial, Helvetica, sans-serif">Welcome. My name is Mohammed, I'm a student and I live in Rafah. On this website, I present photos and reports about my home town. About our life, our community, the home demolitions, homeless families, the children in our camp.. About the tragedies that happen here every day.</font>
</div>
</blockquote>
</td>
</tr>
<tr>
<td bgcolor="#FFFFFF">
<div align="right">
<font size="1" face="Verdana, Arial, Helvetica, sans-serif">Site by Virtual Activism</font>
</div>
</td>
</tr>
</table>
</td>
</tr>
</table>
<h3>�</h3>
<h3><font color="white"></font></h3>
<h3>�</h3>
</body>
</html>
Apache