Найден корректный robots.txt
Файл управления сканированием сайта роботами
?
Robots.txt указывает поисковым роботам, какие страницы сканировать, а какие — нет. Ошибки в файле могут случайно закрыть важные разделы от индексации.
Robots.txt настроен корректно. Размер файла: 12460 байт. Загружен за: 0сек.
Проверяемая страница не запрещена в robots.txt.
Цепочка редиректов для файла robots.txt:
http://codefordemocracy.org/robots.txt
301 MovedPermanently
https://codefordemocracy.org/robots.txt
200 OK
Показать содержимое robots.txt
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0" />
<link rel="icon" type="image/png" href="//assets.codefordemocracy.org/img/favicon.png" />
<link rel="stylesheet" href="//assets.codefordemocracy.org/css/cfd.css">
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.4/css/all.min.css" />
<title>Code for Democracy — Write Code, Defend Democracy</title>
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-142391396-1"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-142391396-1');
</script>
<meta name="description" content="We build search, visualization, and alerting tools that make it easy to investigate relationships between campaign contributions, political narratives, and legislative outcomes in the US. A 501(c)(3) nonprofit.">
<meta property="og:image" content="https://assets.codefordemocracy.org/img/cover.png" />
</head>
<body>
<!-- bar -->
<div class="bg-red">
<div class="container py-3 px-5 mx-auto mb-3">
<p class="text-white text-sm text-center"><span class="font-semibold">Heads up!</span> Our tools will no longer be available after September 30, 2023.</p>
</div>
</div>
<!-- / bar -->
<!-- header -->
<header class="container px-5 py-7 mx-auto">
<nav class="navbar flex flex-wrap justify-between items-stretch">
<div class="flex items-center flex-shrink-0">
<img class="navbar-logo" src="//assets.codefordemocracy.org/img/logo-small.png" width="147" height="62">
</div>
</nav>
</header>
<!-- header -->
<!-- cover -->
<section class="py-10 md:py-20 mb-5 md:mb-12">
<div class="container px-5 mx-auto">
<div class="max-w-4xl">
<p class="text-darker text-3xl md:text-5xl font-extralight leading-snug md:leading-snug mb-7">Our <span class="font-medium">search, visualization, and alerting tools</span> make it easy to investigate relationships between campaign contributions, political narratives, and legislative outcomes</p>
<a href="https://docs.codefordemocracy.org/" target="_blank" class="btn btn-lg btn-primary w-full sm:w-auto mb-1">View Documentation</a>
<button class="btn btn-lg btn-secondary w-full sm:w-auto mb-1" disabled>Launch Our Tools</button>
</div>
</div>
</section>
<!-- /cover -->
<!-- press -->
<section class="py-5">
<div class="container mx-auto flex justify-between sm:justify-end items-center">
<a href="https://www.npr.org/2020/10/05/918520692/facebook-keeps-data-secret-letting-conservative-bias-claims-persist" target="_blank"><img src="img/npr.png" class="mx-3 p-3" width="94" height="68"></a>
<a href="https://www.vox.com/recode/21419328/facebook-conservative-bias-right-wing-crowdtangle-election" target="_blank"><img src="img/vox.png" class="mx-3 p-3" width="83" height="40"></a>
<a href="https://wearecommons.us/the-role-of-open-source-researchers-in-election-security/" target="_blank"><img src="img/commons.png" class="mx-3 p-3" width="117" height="40"></a>
</div>
</section>
<!-- / press -->
<!-- examples -->
<section class="bg-light border-t border-secondary py-10 md:py-20">
<div class="container mx-auto px-5">
<h2 class="text-2xl md:text-4xl font-medium mb-5">What kind of insights can I get?</h2>
<p class="text-md md:text-xl font-light leading-relaxed md:leading-relaxed max-w-3xl mb-9 md:mb-12">Get answers to these questions and more sent straight to your inbox when you set up alerts on our platform. Set up once, stay updated forever.</p>
<div class="grid grid-cols-1 lg:grid-cols-2 gap-10">
<div class="bg-white p-10 rounded-sm filter drop-shadow-sm">
<p class="text-lg md:text-2xl font-light leading-normal md:leading-normal mb-4">Which political candidates are using contributions to their campaigns to give charitable donations?</p>
<button class="btn btn-md btn-gray w-full sm:w-auto mt-1" disabled>View Results</button>
<button class="btn btn-md btn-secondary w-full sm:w-auto mt-1" disabled>Create Email Alert</button>
</div>
<div class="bg-white p-10 rounded-sm filter drop-shadow-sm">
<p class="text-lg md:text-2xl font-light leading-normal md:leading-normal mb-4">Did contributions from defense contractors to election objectors resume after the controversy?</p>
<button class="btn btn-md btn-gray w-full sm:w-auto mt-1" disabled>View Results</button>
<button class="btn btn-md btn-secondary w-full sm:w-auto mt-1" disabled>Create Email Alert</button>
</div>
<div class="bg-white p-10 rounded-sm filter drop-shadow-sm">
<p class="text-lg md:text-2xl font-light leading-normal md:leading-normal mb-4">What types of political contributions are senior executives from big tech companies making?</p>
<button class="btn btn-md btn-gray w-full sm:w-auto mt-1" disabled>View Results</button>
<button class="btn btn-md btn-secondary w-full sm:w-auto mt-1" disabled>Create Email Alert</button>
</div>
<div class="bg-white p-10 rounded-sm filter drop-shadow-sm">
<p class="text-lg md:text-2xl font-light leading-normal md:leading-normal mb-4">Which groups and funding entities are creating Facebook ads about critical race theory?</p>
<button class="btn btn-md btn-gray w-full sm:w-auto mt-1" disabled>View Results</button>
<button class="btn btn-md btn-secondary w-full sm:w-auto mt-1" disabled>Create Email Alert</button>
</div>
</div>
</div>
</section>
<!-- / examples -->
<!-- about -->
<section class="border-t border-secondary pt-10 md:py-20">
<div class="container px-5 mx-auto">
<div class="max-w-4xl mx-auto sm:text-center">
<p class="text-darker text-3xl md:text-5xl font-extralight leading-snug md:leading-snug">We're a 501(c)(3) nonprofit that <span class="font-medium">fights hidden influence in US politics</span> by building tools that automate the analysis of open data</p>
<p class="text-lg md:text-xl font-light leading-relaxed md:leading-relaxed mt-5 md:mt-7 mb-7">Code for Democracy helps journalists, researchers, and open government advocates investigate relationships between campaign contributions, political narratives, and legislative outcomes. Our mission is to strengthen democratic processes by creating digital tools that support free and fair elections, open governance, and fact-based public policy.</p>
<p class="text-lg md:text-xl font-light leading-relaxed md:leading-relaxed mt-5 md:mt-7 mb-7">Code for Democracy started as a volunteer-driven meetup in NYC, SF, DC, and Boston in 2017 and gained its nonprofit status in 2019. We are a non-partisan, values-based organization driven by our commitment to transparency and accountability.</p>
</div>
</div>
</section>
<!-- /about -->
<!-- tribe -->
<section class="pb-20">
<div class="container px-5 mx-auto">
<div class="grid grid-cols-1 lg:grid-cols-3 gap-12">
<p class="text-xl font-extralight leading-relaxed sm:text-center md:text-left">Our work wouldn't have been possible without the early believers who took a chance on us.</p>
<div class="lg:col-span-2">
<div class="grid grid-cols-1 sm:grid-cols-2 gap-5">
<div class="border-l-4 border-secondary pl-5">
<a href="https://www.southparkcommons.com/" class="text-lg text-primary" target="_blank">South Park Commons</a>
<h6 class="text-xs label">Community</h6>
</div>
<div class="border-l-4 border-secondary pl-5">
<a href="https://www.beondeck.com/" class="text-lg text-primary" target="_blank">On Deck</a>
<h6 class="text-xs label">Community</h6>
</div>
<div class="border-l-4 border-secondary pl-5">
<a href="https://www.aaja.org/" class="text-lg text-primary" target="_blank">AAJA</a>
<h6 class="text-xs label">Mentorship</h6>
</div>
<div class="border-l-4 border-secondary pl-5">
<a href="https://www.bakerthomaslaw.com/" class="text-lg text-primary" target="_blank">Baker Thomas Oakley Greene</a>
<h6 class="text-xs label">Law</h6>
</div>
<div class="border-l-4 border-secondary pl-5">
<a href="https://www.hansonbridgett.com/" class="text-lg text-primary" target="_blank">Hanson Bridgett</a>
<h6 class="text-xs label">Law</h6>
</div>
<div class="border-l-4 border-secondary pl-5">
<a href="https://www.presidentialscholars.org/" class="text-lg text-primary" target="_blank">US Presidential Scholars Foundation</a>
<h6 class="text-xs label">Funding</h6>
</div>
<div class="border-l-4 border-secondary pl-5">
<a href="https://ncopengov.org/" class="text-lg text-primary" target="_blank">NC Open Government Coalition</a>
<h6 class="text-xs label">Funding</h6>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- / tribe -->
<!-- count -->
<section class="border-t border-secondary">
<div class="container px-5 mx-auto grid grid-cols-1 md:grid-cols-2 gap-12 md:gap-0">
<div class="pt-12 md:pb-12 md:pr-12 xl:py-20 xl:pr-20 md:border-r border-secondary">
<h6 class="text-7xl font-bold text-red">17M+</h6>
<h5 class="text-2xl font-extralight my-3">Entities</h5>
<p class="text-lg font-light">We process our tracked datasets and extract political candidates, political action committees, political donors, political ad purchasers, and political vendors</p>
</div>
<div class="pb-12 md:pt-12 md:pl-12 xl:py-20 xl:pl-20">
<h6 class="text-7xl font-bold text-red">289M+</h6>
<h5 class="text-2xl font-extralight my-3">Relationships</h5>
<p class="text-lg font-light">We process our tracked datasets and extract both explicit connections based on transactions as well as implicit connections based on metadata, hashtags, and named entities</p>
</div>
</section>
<!-- / count -->
<!-- footer -->
<footer style="background-color: #232323;">
<div class="container py-7 px-5 mx-auto grid grid-cols-1 lg:grid-cols-3 gap-5">
<div class="tracking-widest lg:col-span-2 text-sm flex flex-col sm:flex-row items-center justify-center lg:justify-start lg:-ml-4">
<p class="text-xs uppercase text-muted text-center lg:text-left">© 2017-2023 Code for Democracy. A 501(c)(3) nonprofit. EIN 83-3407325.</p>
</div>
<ul class="col-span-1 text-xl text-muted flex items-center justify-center lg:justify-end lg:-mr-3">
<li class="px-3"><a href="https://github.com/codefordemocracy" target="_blank"><i class="fab fa-github"></i></a></li>
<li class="px-3"><a href="https://twitter.com/CFDorg" target="_blank"><i class="fab fa-twitter"></i></a></li>
<li class="px-3"><a href="https://www.facebook.com/codefordemocracy" target="_blank"><i class="fab fa-facebook"></i></a></li>
<li class="px-3"><a href="https://www.instagram.com/codefordemocracy/" target="_blank"><i class="fab fa-instagram"></i></a></li>
<li class="px-3"><a href="https://www.linkedin.com/company/codefordemocracy/" target="_blank"><i class="fab fa-linkedin"></i></a></li>
</ul>
</div>
</footer>
<!-- / footer -->
<script type="module" src="https://static.cloudflareinsights.com/beacon.min.js/v4513226cdae34746b4dedf0b4dfa099e1781791509496" integrity="sha512-ZE9pZaUXND66v380QUtch/5sE9tPFh2zg45pR2PB0CVkCtOREv2AJKkSidISWkysEuQ0EH8faUU5du78bx87UQ==" data-cf-beacon='{"version":"2024.11.0","token":"6398675a6472451c928642c0cec8cac8","r":1}' crossorigin="anonymous"></script>
</body>
</html>
Cloudflare