# Air Quality Daily. URLs are lowercase and never end in a slash (only the home page is /).
#
# Search and answer bots (Googlebot, Bingbot, OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User,
# PerplexityBot, Perplexity-User, Applebot, DuckDuckBot) follow the rules for every crawler.
# Google-Extended is allowed too, on purpose (owner's call, 2026-10-28). It isn't Google Search or AI Overviews (those
# are Googlebot): it controls Gemini training and Gemini app and Vertex AI grounding. We keep it for Gemini's citations
# and links; the Content-Signal line below still says no to training.
# /find stays crawlable on purpose: its pages say noindex, and a crawler only sees that if it may fetch them.
# /widget/ and /badge/ (embeds on other sites) stay crawlable for the same reason: they answer X-Robots-Tag: noindex,
# and a search engine may still render them inside the pages that embed them.
#
# Content signals (https://contentsignals.org): search=yes lets search engines index and link us; ai-input=yes lets
# AI answers use a page to answer a question and link it; ai-train=no reserves our pages from AI training.
# The rules for using the site, scraping and AI training included: https://airqualitydaily.com/terms#use
User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Disallow: /api/
Disallow: /dev/

# Training-only crawlers and data resellers. They send no visitors.
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
User-agent: Applebot-Extended
User-agent: Meta-ExternalAgent
User-agent: Bytespider
User-agent: Diffbot
User-agent: cohere-training-data-crawler
User-agent: AI2Bot
User-agent: Timpibot
User-agent: omgili
User-agent: omgilibot
User-agent: img2dataset
Disallow: /

# SEO-tool crawlers. They map a site's pages, keywords and links for other sites and send no visitors
# (Search Console and Bing Webmaster Tools cover our own needs).
User-agent: AhrefsBot
User-agent: AhrefsSiteAudit
User-agent: SemrushBot
User-agent: SiteAuditBot
User-agent: MJ12bot
User-agent: DotBot
User-agent: rogerbot
User-agent: DataForSeoBot
User-agent: BLEXBot
User-agent: serpstatbot
User-agent: Barkrowler
Disallow: /

Sitemap: https://airqualitydaily.com/sitemap.xml