Cookie Policy
The website uses essential session and security cookies for core operation. Analytics or advertising cookies are used only if the corresponding services are enabled.
Effective date: July 15, 2026. This Cookie Policy explains how the website uses cookies and similar browser storage when you visit the public website, use its tools, or sign in to the private administration area.
What cookies are
Cookies are small pieces of data that a website asks a browser to store and return with later requests. They can keep a session active, protect forms, remember a choice, measure usage, or support advertising. Some cookies are set directly by this site, while optional third-party services may set their own cookies if those services are enabled.
Essential cookies used by the site
The application uses an essential session cookie to maintain secure interactions with the server. This is particularly important for administrator sign-in, form protection, and actions that must remain connected across more than one request. The session cookie is HTTP-only and uses a same-site setting intended to reduce unwanted cross-site use.
The site may also use a CSRF protection token or related browser value when a form is submitted. This helps the server distinguish a legitimate form request from an unauthorized cross-site request. Essential cookies are required for security and core operation; disabling them may prevent login, form submission, or other interactive features from working correctly.
Public Generator and Checker activity
The public robots.txt Generator and Checker can use the site's normal session and security mechanisms when processing a request. The Checker may also record operational information described in the Privacy Policy, such as the submitted domain, result summary, response status, and a protected hash derived from the request IP address. These records are application data, not advertising cookies.
Analytics cookies
Google Analytics is supported by the application but is loaded only when an analytics measurement ID is configured by the site operator. If analytics is enabled, Google may set or read cookies and process information about visits, pages viewed, device or browser characteristics, approximate location, and interactions. Analytics should be configured together with any consent controls required for the site's audience and operating region.
Advertising cookies
The current public layout does not automatically load an advertising script. If advertising technology is added and enabled later, the provider may use cookies or similar technologies for ad delivery, frequency control, fraud prevention, measurement, or personalization. Any such activation should be reflected in this policy and accompanied by the consent controls required for the site's audience and operating region.
How to manage cookies
You can inspect, delete, or block cookies through your browser settings. Most browsers also allow you to block third-party cookies or clear site data for a single domain. Blocking essential cookies may break administrator access and interactive forms. Blocking optional analytics or advertising cookies should not prevent you from reading the public content.
Changes to this policy
This policy may be updated when the site's configuration, analytics, advertising, or legal requirements change. The effective date at the top will be revised when a material update is published. Questions about this policy can be sent through the Contact page.
Current cookie and consent controls
Consent controls effective: July 16, 2026.
The site uses a global opt-in model for optional technologies. Analytics and advertising are denied by default and are not loaded through the built-in integration until a visitor makes an affirmative choice. “Accept all” and “Reject non-essential” are available on the first layer with equal prominence.
Change or withdraw your choice
You can reopen the privacy center at any time. Withdrawing permission updates Google consent signals immediately, stops future optional loading through the site integration, and removes known first-party Google Analytics cookies where the browser allows it.
Global Privacy Control
When the browser sends a Global Privacy Control signal, the site treats it as an opt-out from advertising-related sale, sharing, and cross-context behavioral advertising. That choice overrides an earlier advertising permission while the signal is active.
Google advertising consent
The built-in privacy center is designed to control the site’s optional scripts, but it is not an IAB TCF or Google-certified consent management platform. Before Google AdSense is activated for visitors in the EEA, the United Kingdom, or Switzerland, the site must additionally use Google Privacy & Messaging or another Google-certified IAB TCF platform.
FAQ
Which cookies are essential on this site?
The application may use a session cookie and CSRF-related protection for secure forms, administrator authentication, and other interactions that must stay connected across requests.
Does the site currently require analytics cookies?
No. Analytics support is optional and the public layout loads it only when a measurement ID has been configured by the site operator.
Can I block all cookies?
You can block or delete cookies in your browser, but blocking essential cookies may prevent login, form submission, or other interactive features from working correctly.
Are advertising cookies always active?
No. The current public layout does not automatically load an advertising script. If advertising technology is added later, this policy and any required consent controls should be updated.
Where can I ask a privacy question?
Use the Contact page for questions about cookies or personal data, and read the Privacy Policy for more information about application logs and submitted tool data.
Related tools
Related pages
Privacy Policy
Robots.txt Checker for AI Crawlers
The website processes the domains submitted to its checker and may retain a one-way IP hash, domain, response status, and summary results for diagnostics and abuse prevention. Optional analytics or advertising operate only when configured.
Terms of UseThe website provides technical information and tools on an as-available basis. You are responsible for reviewing generated rules, protecting private content with real access controls, and using the service lawfully without abuse.
ContactUse the email below for technical questions, corrections, crawler documentation updates or problems with the site.
About this websiteThe website is an independent technical resource that helps site owners understand crawler purposes, generate selective robots.txt rules, test live files, and avoid confusing crawler policy with real access security.
Robots.txt Checker for AI CrawlersEnter a domain, choose a crawler, and test a path. The checker downloads the live robots.txt file, finds the applicable User-agent group and most specific rule, then explains whether access is allowed or blocked.