improve robots.txt, update sitemap.xml
Search engines become confused when they discover pages such as under `inc/head.default.html` that are **NOT** real pages, but rather quasi-templates that are used by bin tools in the generation of HTML content. So we "_Disallow_" search engine access to URLs under `inc/` except for the two "safe" pages. Updated `sitemap.xml` as per recent modifications.
This commit is contained in:
1 parent
49068e4c5b
commit
84017fb0c7
2 files changed
+45
-43
No files matched your search
@@ -1,3 +1,5 @@
|
||||
User-Agent: *
|
||||
Allow: /inc/index.html
|
||||
Disallow: /inc/
|
||||
Allow: /
|
||||
Sitemap: https://www.ioccc.org/sitemap.xml
|
||||
Reference in new issue
Block a user