# Allow all good bots to index the site (except sensitive areas) User-agent: * Disallow: /private/ Disallow: /tmp/ Disallow: /admin/ Disallow: /404.html Disallow: /*?* Allow: /css/ Allow: /js/ Allow: /img/ # Block known spammy bots that abuse sitemaps or scan sites User-agent: MJ12bot Disallow: / User-agent: SemrushBot Disallow: / User-agent: dotbot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: BLEXBot Disallow: / # Your sitemap location Sitemap: https://meknika.com/sitemap_new.xml